Redact Processor

Description

With this Processor you can identify and redact Personally Identifiable Information (PII) in your data stream. This includes items like:

Social Security numbers
Email addresses
Credit cards
Phone numbers

This Processor identifies PII based on pattern detection. When the specified pattern is detected, you have the option to replace it with a specified string, or use a standard hash. When using this Processor in a Mezmo Edge Pipelines for Local Data, you can also specify a regular expression to use for pattern detection.

Configuration

Option	Description	Example/Options
Title	A title for the processor	Redact SSN
Description	A description of the Processor’s function in the Pipeline	Replaces SSNs with a text string
Field (Optional)	The field to search for the pattern. If the field is not specified, the Processor will look for the pattern across all the fields.	`Message`
Mask Pattern
	Pattern	Credit card number Email address IPv4 address US or Canada phone number US social security number Custom Regex Pattern (Mezmo Edge Pipelines for Local Dataonly)
	Action	Hash Replace Detect Only
	Hash Options The value is hashed and base64 encoded	md5 sha1 sha2 (sha2-512_256) sha3 (sha3-512)
	Replacement	<Any text string>
PII Presence	When pre-set or custom PII patterns are detected in an event, a metadata field will be added to the event that can be used by down stream processors for collecting metrics or trigger an alert that this pattern is present. Metadata field will indicate one or more PII patterns detected in the event as shown below: “pii_presence”: { “email_address”: “yes”, “us_social_security_number”: “yes” } If no PII patterns are detected, pii_presence field will not be added to the metadata.	On or Off

Pre-defined Patterns

The US social security number pattern matches any valid social security number sequence, with or without optional delimiters. Examples:

XXX YY ZZZZ
XXX-YY-ZZZZ
XXX YY-ZZZZ
XXX-YY ZZZZ
XXXYYZZZZ

Phone Number

The phone number pattern matches valid phone numbers, with or without space, hyphen or period (.) delimiters. Valid phone numbers include:

+1 XXX YYY ZZZZ
+1 (XXX) YYY ZZZZ
+1.(XXX).YYY.ZZZZ
+1XXXYYYZZZZ
+1-XXX YYY.ZZZZ, etc. .

The matcher will also match parts of sequences (false positives). Partially matched phone numbers includes:

XXX.YYY.ZZZZ will be matched in the input +1_XXX.YYY.ZZZZ
XXX.YYY.ZZZZ will be matched in the input +1.MXXX.YYY.ZZZZ

Credit Card Number

The credit card pattern currently matches Visa, MasterCard, American Express, Diners club and JCB card numbers. Visa 13 to 16 digit numbers starting with 4. Examples: 4XXXXXXXXXXXX or 4XXXXXXXXXXXXXXX MasterCard 16 digit numbers starting with 21-27 (new range) or 51-57 (old range). Examples: 21XXXXXXXXXXXXXX or 56XXXXXXXXXXXXXX American Express (AMEX) 15 digit numbers starting with 34 or 37. Examples: 34XXXXXXXXXXXXX or 37XXXXXXXXXXXXX Diners Club 14 digit numbers starting with 300-305 or 360-389. Examples: 300XXXXXXXXXXX or 389XXXXXXXXXXX JCB 15 digit numbers starting with 2131 or 1800 or 16 digit numbers starting with 35. Examples: 2131XXXXXXXXXXX.

Email Address

Matches RFC 5322 compliant email addresses. Examples: user@dummy.com, user@127.0.0.1, etc

Interactive Demo

Check out an interactive demo of the Redact Processor as a component in a Compliance group, as well as instructions for building a version of the Pipette with your own sample data.

Examples

Option	Value
Pattern	US social security number
Action	Replace
Replacement	000-00-000

Option	Value
Pattern	US social security number
Action	Hash
Hash	md5

Option	Value
Pattern	Custom
Action	Replace
Expression	(?<ssn_canada>(\d{3})-(\d{3})-(\d{3}))
Replacement	111-111-111

Redact Email Address to String

Option	Value
Pattern	Email address
Action	Replace
Replacement	someone@nowwhere.com

⌘I

About Pipelines

Sources

Processors

Destinations

Pipeline Management

Other Topics

Description

Configuration

Pre-defined Patterns

Phone Number

Credit Card Number

Email Address

Interactive Demo

Examples

Redact Email Address to String

​Description

​Configuration

​Pre-defined Patterns

​Social Security Number

​Phone Number

​Credit Card Number

​Email Address

​Interactive Demo

​Examples

​Redact Social Security Number with String

​Redact Social Security Number to Hash

​Custom Regex Example: Redact Canadian Social Security Number

​Redact Email Address to String

Description

Configuration

Pre-defined Patterns

Social Security Number

Phone Number

Credit Card Number

Email Address

Interactive Demo

Examples

Redact Social Security Number with String

Redact Social Security Number to Hash

Custom Regex Example: Redact Canadian Social Security Number

Redact Email Address to String