What Changed

Huntington Bank reduced the processing time of over 400M documents from several years to just a few months by leveraging AWS’s toolkit. This change was achieved by combining services such as Amazon Textract, Amazon SageMaker, AWS Step Functions, and AWS Lambda. (Source: Huntington Bank: Redacting sensitive data from 400M+ documents with AWS)

Detailed Mechanism

The architecture of this solution consists of four stages: data transfer from on-premises file sharing to Amazon S3, machine learning-based data extraction using Textract, workflow management using Step Functions, and event-driven processing using Lambda. AWS DataSync and AWS Direct Connect were utilized to migrate over 400M documents in an encrypted state. Textract extracts text and tables from scanned documents, and SageMaker trains a custom model to detect sensitive data. (Source: Huntington Bank: Redacting sensitive data from 400M+ documents with AWS)

Migration Procedure

  1. Data Migration: Deploy AWS DataSync on-premises and set up data transfer from SMB file sharing to an Amazon S3 bucket.
  2. Document Processing: Use Amazon Textract to scan documents and extract sensitive information using a machine learning model.
  3. Workflow Management: Define the processing flow using AWS Step Functions and execute event-driven processing using Lambda functions.
  4. Security Assurance: Manage encryption using AWS KMS and ensure the security of data in transit and at rest. (Source: Huntington Bank: Redacting sensitive data from 400M+ documents with AWS)

Performance Characteristics

The time required for processing was reduced from several years to just a few months by leveraging AWS’s toolkit. Although specific performance numbers are not mentioned in the official documentation, the scalability and efficiency of large-scale document processing are emphasized. (Source: Huntington Bank: Redacting sensitive data from 400M+ documents with AWS)

Summary

  • Automate large-scale document migration using AWS DataSync and Amazon S3, ensuring security.
  • Extract sensitive information using Amazon Textract and SageMaker, improving accuracy by training custom models.
  • Build event-driven workflows using AWS Step Functions and Lambda, enabling process efficiency and cost reduction.
  • Enhance data security using AWS KMS encryption management for data in transit and at rest.
  • Refer to official documentation (AWS DataSync, Amazon Textract) and consider the optimal settings for your environment.