WebSocket Entity Data API (1.0.1)

This application handles dynamic and potentially long-running queries with a robust and scalable infrastructure. The flow includes authentication, queuing, data processing, and asynchronous communication back to the client.

Dev Base Url: wss://ws-dev.iiris.com

Prod Base Url: wss://ws.iiris.com

Authentication

The authentication mechanism of the Entity Data API utilizes AWS Cognito as the underlying authentication service. AWS Cognito provides a robust and secure user authentication and authorization solution that integrates seamlessly with various applications and platforms. In the context of the Entity Data API, the authentication process involves creating an access token using the Cognito authentication endpoint, client credentials, and scope.

Access Token: An access token is a credential that is used to authenticate and authorize access to protected resources. In the context of the Entity Data API, the access token is generated by AWS Cognito and serves as proof of the user's identity and permissions. It acts as a bearer token, allowing the user to access the Entity Data API's protected resources.

Cognito Auth Endpoint: The Cognito authentication endpoint is the URL provided by AWS Cognito that developers can use to authenticate users and obtain access tokens. This endpoint handles the authentication process, verifies user credentials, and issues access tokens upon successful authentication.

Dev Auth Endpoint: https://idp.dev.iris.informa.com/oauth2/token

Prod Auth Endpoint: https://idp-internal.iris.informa.com/oauth2/token

Client Credentials: The client credentials refer to the unique identifier and secret associated with the client application that is integrating with the Entity Data API. These credentials are typically generated when the client application is registered with AWS Cognito. They serve as the client's authentication credentials when requesting an access token from the Cognito authentication endpoint. These credentials will be shared with partners separately.

The authentication mechanism of the Entity Data API using AWS Cognito follows these steps:

The client application initiates the authentication process by sending a request to the Cognito authentication endpoint, providing the necessary parameters, including the client credentials and desired scope. AWS Cognito verifies the client credentials and scope, ensuring that the client application is authorized to request an access token with the specified permissions. Upon successful verification, AWS Cognito generates an access token, which is returned as a response to the client application. The client application receives the access token and includes it in subsequent requests to the Entity Data API as an authorization header. This access token acts as proof of the client's authentication and authorization, allowing access to the protected resources. The Entity Data API validates the access token by verifying its authenticity, checking its expiration, and ensuring that it has the required permissions to access the requested resources.

Fetch: getEntitydata route for Entity Data Retrieval

The Entity Data route provides a flexible and dynamic interface for retrieving data from various logical tables. This approach allows clients to specify multiple parameters and operators for efficient data retrieval.

Below is a summary of the logical tables and their corresponding physical table names, along with the available operators for filtering:

Logical Name	Physical Table Name
Messages	unified_messages_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status product_instance_code product_instance_name audience_id email message_direction action_date message_id entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
ContentViews	unified_contentview
View Columns source_system source_system_instance primary_data_source record_status unique_key user_id email first_party_tracking_id third_party_tracking_id session_tracking_id content_type content_class content_category td_url content_url_title content_url_host content_url_path content_url_query_string content_url_fragment activity_timestamp engagement_time utm_campaign utm_content utm_medium utm_source utm_term operating_system_name device_brand device_version device_class ip_address ip_country ip_city ip_latitude ip_longitude ip_province ip_time_zone ip_postal_code browser_language browser_viewport_width browser_viewport_height browser_horizontal_pixels_scrolled browser_vertical_pixels_scrolled browser_horizontal_percentage_scrolled browser_vertical_percentage_scrolled referrer_url referrer_url_host referrer_url_path referrer_url_query_string referrer_url_fragment referrer_medium referrer_source referrer_term is_robot robot_category robot_impact robot_reason user_agent application_id page_view_id entitlement_tag_ids updated_timestamp refr_first_party_tracking_id merged_refr_and_first_party_tracking_id content_author content_source gating_level page_desc primary_term auxiliary_terms company_type company_name is_login time
DataPrivacyFlags	unified_data_privacy_flagged
View Columns casl_flag time
Audience	unified_audience
View Columns email email_domain domain_type email_hashed first_name local_firstname last_name local_lastname company_name local_companyname company_size company_type industry job_level job_title job_function mobile_phone other_phone country province city postal_code address facebook_id instagram_id linkedin_id twitter_id snapchat_id entitlement_tag_ids audience_profile first_party_tracking_id third_party_tracking_id updated_timestamp inserted_timestamp last_engaged_timestamp time
BadgeScan	unified_badgescan_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status product_instance_code product_instance_name audience_id email target_entity target_name target_id scan_date topics referral_code entitlement_tag_ids updated_timestamp scan_location product_code product_name brand_code brand_name family_code family_name time
SearchHistory	unified_searchhistory_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status product_instance_code product_instance_name audience_id email source_url search_type search_string search_topics search_date entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
MeetingInvites	unified_meetinginvites_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status product_instance_code product_instance_name audience_id email invite_direction action_date invite_status meeting_id meeting_name meeting_date meeting_status topics referral_code entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
ContentDownload	unified_contentdownload_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status product_instance_code product_instance_name audience_id email identity_source user_id tracking_id content_type content_class content_id content_name content_url content_category content_tags content_sponsor activity_timestamp utm_source utm_medium utm_campaign utm_term utm_content referrer entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
Favourites	unified_favourites_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status product_instance_code product_instance_name audience_id email favourite_direction favourite_target action_date favourite_toggle entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
ObjectViews	unified_object_views_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status product_instance_code product_instance_name audience_id email object_view_type object_view_target_id object_view_target action_date entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
OtherFavourites	unified_other_favourite_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status product_instance_code product_instance_name audience_id email favourite_type favourite_target favourite_topics action_date favourite_toggle entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
AccountActivity	unified_audience_account_activity_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status audience_id email product_instance_code product_instance_name account_activity_type activity_date entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
Subscription	unified_subscription_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status product_instance_code product_instance_name audience_id email product_offering_code product_offering_name product_offering_type subscription_status start_date end_date quantity price price_rate_code subscription_id promotional_code entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
Leads	unified_leads_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status audience_id email lead_status lead_type action_date lead_owner entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
Consent	unified_consent_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status audience_id email consent_type consent_status action_date entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
Session	unified_session_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status audience_id email session_id session_start_time session_end_time action_date entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
Transaction	unified_transaction_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status audience_id email transaction_id transaction_amount transaction_date transaction_status entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
Preference	unified_preference_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status audience_id email preference_type preference_value action_date entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
ProductPurchase	unified_productpurchase_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status audience_id email product_instance_code product_instance_name purchase_date purchase_amount transaction_status entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
Registration	unified_registration_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status audience_id email registration_date registration_status entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
SubscriptionExtension	unified_subscription_extension_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status audience_id email subscription_id extension_start_date extension_end_date action_date entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
Extension	unified_audience_extension_signal
View Columns source_system source_system_instance primary_data_source unique_key record_status audience_id email extension_id extension_start_date extension_end_date action_date entitlement_tag_ids updated_timestamp product_code product_name brand_code brand_name family_code family_name time
LastEngagedProduct	unified_last_engaged_signal_product_union
ViewColumns product_code product_name type last_engaged_timestamp first_engaged_timestamp engaged_kema engaged_3m engaged_6m engaged_1y engaged_2y engaged_3y time
LastEngaged	unified_last_engaged_signal_union
ViewColumns source_system source_system_instance type last_engaged_timestamp engaged_kema engaged_3m engaged_6m engaged_1y engaged_2y engaged_3y engagement_score time
Emailactivity	unified_emailactivity_signal
ViewColumns time source_system source_system_instance unique_key record_status email audience_id activity_type activity_target activity_topics campaign_code campaign_name activity_date total_count unique_count engagement_score entitlement_tag_ids updated_timestamp iiris_td_person_id

Available Operators

These operators can be used in the request body for filtering conditions. Each operator allows you to specify the relationship between a field and its value during data retrieval operations. Make sure to choose the appropriate operator that reflects the comparison you want to perform in your query.

• Equals To
• Not Equal To
• Greater Than
• Less Than
• Greater Than or Equal To
• Less Than or Equal To
• Is Null
• Is Not Null
• In
• Between
• Exists
• Like
• ILIKE

WebSocket Endpoint:
dev- wss://ws-dev.iiris.com prod - wss://ws.iiris.com

Include the authorizationToken as a query parameter for authentication.

Example Connection URL:
wss://ws-dev.iiris.com?authorizationToken=Bearer <your_token>

Replace <your_token> with a valid authorization token.

---

/connect

• Purpose: Establish a WebSocket connection and authenticate the client.
• Description:
  Use this route to initiate a WebSocket connection. A valid authorizationToken must be provided as a query parameter.

Responses:

• 200 OK: Connection successfully established.

  • Description: The WebSocket connection is open, and authentication is successful.

• 401 Unauthorized: Invalid or missing authorizationToken.

  • Description: The provided authorizationToken is invalid or missing in the query parameters.

• 403 Forbidden: The authorizationToken has expired.

  • Description: The provided authorizationToken has expired, and the connection cannot be established.

• 500 Internal Server Error: An unexpected error occurred on the server.

  • Description: An internal server error occurred, preventing the WebSocket connection from being established.

---

/getEntityData

The Entity Data route provides a flexible and dynamic interface for retrieving data from various logical tables. This approach allows clients to specify multiple parameters and operators for efficient data retrieval.

Request Body Schema: application/json

The request body is a JSON object that must include the following fields:

• verbose (optional, default "N"):
  Indicates whether detailed information should be included in the response.

  • "Y": Returns verbose data with detailed fields. (Returns intermediate messages from real-time updates.)
  • "N": Returns basic data with minimal fields. (Returns only streamed response data.)

• compress (optional, default "N"):
  Specifies whether the response should be compressed.

  • "Y": The response will be compressed to reduce payload size.
  • "N": The response will not be compressed.

• use_case (required):
  A string representing the context or purpose of the request (e.g., "marketing_campaign").

• campaign_id (optional):
  A string representing the campaign ID (e.g., "ABC123"). If not provided, the field can be null.

• filters (required):
  An object containing dynamic criteria for filtering the data. The filter object must include:

  • logic (string): Defines the logic to apply between conditions (e.g., "AND", "OR").
  • conditions (array): An array of filter conditions, where each condition is an object containing:
    • field (string): The field to filter on (e.g., "Audience.domain_type").
    • operator (string): The operator to use for filtering (e.g., "Equals To", "Like").
    • value (string): The value to compare the field against.

• operation (required):
  A string defining the operation type (e.g., "export" or "count").

  • If export is chosen, the response will include detailed data records.
  • If count is chosen, the response will return only the total number of records without the data details.

---

Example Request

Below is an example of a request message sent via WebSocket:

{
    "action": "getEntityData",
    "queryStringParameters": {
        "verbose": "Y",
        "compress": "N",
        "use_case": "marketing_campaign",
        "campaign_id": "ABC123",
        "filters": {
            "logic": "AND",
            "conditions": [
                {
                    "field": "Audience.domain_type",
                    "operator": "Equals To",
                    "value": "External"
                },
                {
                    "field": "Audience.company_name",
                    "operator": "Like",
                    "value": "%A"
                }
            ]
        },
        "operation": "export"
    }
}

Request Flow:

1. Initiate Request: The request is received, and the connection ID is sent back to the client.
2. Background Processing: The query is processed in the background.
3. Real-time Updates: The client will receive real-time updates via WebSocket as the data is being processed.
4. Final Response:
   • If data is found, the client receives the final data stream.
   • If no records are found, the client receives a "no data found" message.
   • In case of an error, a 500 Internal Server Error will be returned.

Responses:

{
    "Response Chunk 149/149": {
        "connectionID": "BjffodzCDoECI2A=",
        "requestId":"BjffodzCDoEabcd=",
        "data": [
            {
                "iiris_td_person_id": "23-*****",
                "email": "*****@****.ky",
                "email_domain": "****.ky",
                "domain_type": "External",
                "email_hashed": "********",
                "first_name": "Gaone",
                "local_firstname": "",
                "last_name": "Dube",
                "local_lastname": "",
                "company_name": "CIMA",
                "local_companyname": "",
                "company_size": "",
                "company_type": "",
                "industry": "",
                "job_level": "",
                "job_title": "",
                "job_function": "",
                "mobile_phone": "",
                "other_phone": "",
                "country": "United States of America (the)",
                "province": "",
                "city": "",
                "postal_code": ".",
                "address": "",
                "facebook_id": "",
                "instagram_id": "",
                "linkedin_id": "",
                "twitter_id": "",
                "snapchat_id": "",
                "entitlement_tag_ids": "*****",
                "audience_profile": "Known",
                "first_party_tracking_id": "[]",
                "third_party_tracking_id": "[]",
                "updated_timestamp": 1680193073,
                "inserted_timestamp": 1680193073,
                "last_engaged_timestamp": 1680193073,
                "time": 1701813600
            }
            }
        ]
    }
}

• 200 OK: Request successfully initiated.

  • Description: The request has been initiated successfully, and the client will receive a connection ID.
  • Message: "Request initiated successfully. You will receive connection ID shortly."

• 500 Internal Server Error: An unexpected error occurred on the server.

  • Description: The server encountered an issue while processing the request, preventing the data retrieval process from completing.
  • Message: "Internal Server Error: Unable to process the request."

• No Data Found (Final Response, optional based on query result):

  • Description: If no records match the query, the client will receive a message indicating no data was found.
  • Message: "No data found for the given query."

Incremental Pull

The Incremental Pull feature in the EntityData API allows you to fetch only the latest records based on a previously executed query. For example, if you have already streamed records for the past 60 days using the EntityData API, and after 10 days you re-execute the same query with a specific parent_request_id, the API will return only the records for the last 10 days. This helps in efficiently retrieving new or updated data without fetching all the records again.

Example Request

Below is an example of a request message sent via WebSocket:

{
    "action": "getEntityData",
    "queryStringParameters": {
      "parent_request_id":"HT-6mGAIDoEEkRA=",
        "verbose": "Y",
        "compress": "N",
        "use_case": "marketing_campaign",
        "campaign_id": "ABC123",
        "filters": {
            "logic": "AND",
            "conditions": [
                {
                    "field": "Audience.domain_type",
                    "operator": "Equals To",
                    "value": "External"
                },
                {
                    "field": "Audience.company_name",
                    "operator": "Like",
                    "value": "%A"
                }
            ]
        },
        "operation": "export"
    }
}

• If wrong parent_request_id is provided, the API will return "Parent request ID not found. Please provide a valid request ID."
• If no new records present for query, API will return "No new data available for this parent request id: HT-6mGAIDoEEkRA="

---

/disconnect

Disconnects the WebSocket connection and closes the active session.

Request Flow:

1. Initiate Disconnect: The client sends a message with the action disconnect.
2. Connection Closure: The WebSocket server will close the connection and terminate the session.

Responses:

• 200 OK: Disconnect successful.

  • Description: The WebSocket connection has been successfully closed, and the session is terminated.
  • Message: "Connection closed successfully."

• 500 Internal Server Error: An error occurred while attempting to disconnect.

  • Description: The server encountered an issue when trying to close the WebSocket connection. This could be due to network issues, server errors, or connection timeouts.
  • Message: "Internal Server Error: Unable to close the connection."

After streaming the data, all records are stored in the stream_data table for audit purposes. The table includes a column last_updated_date.

• If any records have a last_updated_date more than 90 days and there is no child request for those records, they will be archived.
• If the record has a child request and the last_updated_date is more than 180 days, the child and parents records will be archived.

Note:-

• After archival of records, child requests cannot be made. A new request must be created.
• The archived records will be stored in an S3 bucket at path: archive/{parent_request_id}/{date}/result.