Splunk : Transform data further from already transformed data

In many circumstances, you may need to extract or transform a data which has just been extracted by another transform. Please see a below case whereby the _raw needs to have a new sourcetype (index time), then on such modified sourcetype you need to extract the fields , and one of the extracted fields itself is an XML.

We can achieve all this using props.conf and transforms.conf within an app (or local/ directory of your existing app)

In props.conf

 # =================================================================
 # These are executed in the same order that they appear in the list so ORDER CAREFULLY!
 [incoming_sourcetype]
 TRANSFORMS-sourcetype = rename_mySourcetype

 [mySourcetype]
 # Search Time extractions by REPORT
 REPORT-mySourcetype = my_deep_extraction_1, my_deeper_extraction_2, my_deeper_extraction_3
 # =================================================================

 In transforms.conf

 # =================================================================
[rename_mySourcetype]
DEST_KEY=MetaData:Sourcetype
REGEX = someSpecialRegexHere
FORMAT = sourcetype::mySourcetype

[my_deep_extraction_1]
DELIMS = ","
FIELDS = "fielda_with_complex_structure","field2","field3"

[my_deeper_extraction_2]
SOURCE_KEY=fielda_with_complex_structure
REGEX=(?<deep1_1>.*)

# Below both are same as above, but multiple ways to do extraction
[my_deeper_extraction_3]
SOURCE_KEY=fielda_with_complex_structure
DELIMS = ","
FIELDS = deep2_1, deep2_2, deep2_3

[my_deeper_extraction_3]
SOURCE_KEY=fielda_with_complex_structure
REGEX = ^\/(?:[^\/]+\/){4}([^\/]+)
FORMAT = deep3_1::$1

 # =================================================================