If the steps of file processing are common across all types and are well defined, as a best practice it is important to have one template/dynamic flow process files.
For example, you have a Claims file that is a csv format and needs to be processed into a tab delimited file (common format). However, you can build a unique flow to handle this file but the more efficient way would be to create a dynamic flow that will process not only this type of file but any other type that needs to be processed as part of the common steps.
A unique flow would have activities like this:
- A dynamic flow would look like this:
- A unique flow can have individual activities also get dynamically overridden but for the purpose of this scenario we will assume that we are overriding everything after the source activity.
- To build a dynamic flow we need to look at the following factors:
§ To create the flow, open the process designer and first design the flow such as:
- Start > Source > Source Schema > Mapping > Target Schema > Target > End
- File Event would trigger of the flow.
- Remember the schema means meta-data or data structure of the file.
- As part of the Process Flow design, one of the steps that we need to add would be to do a lookup into a table based on a file name pattern and fetch related activity ids to be used in the process.
- Lookup table can have as many columns that are in the flow such as FileName, SourceSchemaID, MappingID, TargetSchemaID, TargetID. It can be created in any database such as sql server. Make all varchar types and non-nullable.
- Now one of the new steps in the above flow would be to add a step after the source which will do a lookup into a table based on a file name (use ‘like’ in where condition). This can be a custom java code (one of the links in support forum has a code sample) or it can be a mapping. I prefer the mapping activity.
- The other activity after the lookup would be to assign the Activity IDs to the related steps in the process. We would use put-context-var to do that part. And that is all is needed to make the flow dynamic.