New feature: Amazon Bedrock Data Automation now supports blueprint instruction optimization for documents. Improve Data Automation custom output accuracy by providing example content assets with ground truth labels, enabling you to achieve production-ready

Optimize your blueprints with ground truth

Improve blueprint accuracy by providing example content assets with the correct expected results. Blueprint instruction optimization uses your examples to refine the natural language instructions in your blueprint fields, which improves your inference accuracy.

Blueprint instruction optimization works best when you need to extract specific values that appear directly in your documents, such as invoice numbers, contract amounts, or tax form fields. We recommend providing 3 to 10 example assets that represent the documents you process in production, especially ones where you have encountered accuracy challenges.

How blueprint instruction optimization works

Blueprint instruction optimization analyzes the differences between your expected results and the initial inference results. The service iteratively refines the natural language instructions for each field of your blueprint until the instructions produce more accurate results across your example assets. This process completes in minutes without requiring any model training or fine-tuning.

What you need before you start optimizing your blueprints

• A blueprint with defined fields. Create a blueprint using the console or API. Your blueprint should include the field names and initial descriptions for the data you want to extract.
• Example content assets. Gather 3 to 10 document assets that represent your production workload on documents. Choose examples that contain all the fields in your blueprint.
• Expected results for your examples. Prepare the correct values you want to extract from each example asset. You can enter these values manually during optimization or upload them using a manifest file.
• An S3 bucket location. Specify an S3 bucket where you want to store your example assets and ground truth data. You can provide your own bucket or allow the service to create one for you.

Step-by-step process to optimize your blueprint

1. Select Optimize blueprint to begin the optimization workflow.
2. Upload your example assets. Choose up to 10 content assets from your local device or from an S3 location. The service uploads your assets and displays thumbnails for each file.
3. Provide ground truth for each asset. Select an asset to open the ground truth editor. Enter the correct value you expect to extract in the Ground Truth column.
4. To speed up ground truth entry, select Auto-populate to run initial inference on your assets and automatically populate the Ground Truth column from values in your Results column. Edit any incorrect values before proceeding.
5. Start optimization. After you complete ground truth entry for all selected assets, choose Start optimization. Data automation analyzes your examples and refines the natural language instructions for each field.
6. Review the evaluation metrics. When optimization completes, the Metrics section displays accuracy metrics for your blueprint. Review the overall F1 score, confidence score, and exact match rate to assess whether the blueprint meets your accuracy requirements.
7. Complete optimization. If the evaluation metrics meet your requirements, select Save optimized blueprint to promote the optimized blueprint to production.

Re-optimize your blueprint

You can re-optimize a blueprint at any time to improve accuracy further. Return to the blueprint detail page and select Optimize blueprint. The service displays the assets you previously used for optimization along with their ground truth values.

Edit a blueprint after optimization

If you add or remove fields from an optimized blueprint, the service removes the optimization history and associated example assets. Before editing, download the manifest file that contains your asset locations and ground truth labels.

If you need further guidance on AWS, our experts are available at AWS@westloop.io. You may also reach us by submitting the Contact Us form.