Consuming SAP HCM and AD

4E syncs data from various SAP modules (HCM) on a configured schedule

There are 2 main steps involved in syncing the SAP HCM data into 4E. 

1. Sync SAP HCM data as a generic module into formSubmissionData collection. 

2. Process the formSubmissionData – Map, save/update into 'users’ collection. 

Step 1 - Sync SAP HCM data as a generic module 

A sqs-queue-handler lambda function inside tqmi-sap repository connects to SAP OData service based on provided parameters. It syncs data from different SAP modules. 

This handler handles 3 different types of messages (message type is passed into the request payload of sqs queue) –  

1. GET_SAP_DATA 

2. SAVE_SAP_DATA_INTO_FORM 

3. UPDATE_SAP_DATA 

There are multiple mongodb collections used to store SAP configuration data, which are – 

1. sap_config – This collection has data related to different SAP module. 
   A document in this collection contains some information related to a specific SAP module. Below are some of the important fields -  

2. service_resource – url/endpoint of SAP module, used to connect with that specific module. 

3. service_query_params – query parameters used to fileter the data 

4. service_response_map – Mapping of SAP module fields to 4E fields. 

5. dataCaptureFrequency – reference of dataCaptureFrequency document which contains SAP module data sync frequency schedule. 

6. deltaTokenString – A token for delta sync up. 

7. sap_form_connector – This is a mapping of sap_config and a form, also stores mapping of sap field to form field. 

8. metricCollectionForms – This collection is used to store the forms related data. e.g., formElements, permissions (who can read, edit and fill the form). Also, some information on who created the form, form name and form data source. 

9. metricCollectionFormsData – Has values of metrics in a form. Some important fields are metric key, value, label, UoM, submissionId, capture timestamp. 

How does SAP HCM sync work?

1. Fetch the sap_config record using sapConfigId passed in the request payload. Also get the form for which data needs to be synced. 

2.  Connect to SAP ODATA service using endpoint, endpoint username and password provided in the request. 

3. Update some information like – lastSyncTimestamp, deltaToken (if we get that in the response) 

4. Go through the SAP response map and get the matching data to create/update record in mongodb collection. 

5. Map form fields to SAP response and update push it into formSubmissionData. 

6. If the call is for delta token change, then retrieve the matching record from mongodb formSubmissionData collection and update it. If the record is not present in collection, then create a new one. 

7. Send the records for creation/updation to sqs queue with message type -SAVE_SAP_DATA_INTO_FORM 

8. If it is an update record, then an event will be sent to update the SAP data (event event name UPDATE_SAP_DATA) 

9. If sync fails, then sms and email notifications will be sent using notification templates. 

SAVE_SAP_DATA_INTO_FORM – This handles the creation of new form submission data. 

Once the forms data is updated, an event is sent to sqs queue to calculate/re-caculate the KPI actuals (event name - CALCULATE_KPI_ACTUALS_FOR_SAP_FORM_DATA) 

UPDATE_SAP_DATA – This event handles the update of form data. After this operation also the same event is published to calculate the KPI actuals. 

How to find configuration related to any module from sap_config collection?

Execute this query - {service_resource: {$regex: '/*module_name'}} 

e.g. - {service_resource: {$regex: '/*HCM'}} 

Step 2 - Process and save the users into ‘users’ collection

Once the user data is synced into formSubmissionData another handler processes that data, there is a handler sap-users-integration-handler from tqmi-user-api repository which handles the processing (mapping and storing) operations on raw SAP user data.  

Below are steps which are performed by this handler –  

1. Retrieve userDataForm from userDataForms collection. 

2. Retrieve formSubmission data using the formId of the userDataForm retrieved in above step, which is nothing but the SAP users’ data. 

3. An event COMPARE_FORM_RECORD_AND_UPDATE is raised for each record which compares an existing record and updates. 

4. sqs-queue-handler.ts has a handler function which handles the above event. It gets all the existing users from ‘users’ collection. 

5. Maps 4E fields with SAP fields 

6. Compare all the fields of this user with existing user and updates only updated fields like department, supervisor etc. 

Fields synced from SAP HCM module