I'm trying to automate the ingestion of data from Pub/sub to Bigquery. I need help writing a cloud function to subscribe to a pub/sub, do some processing on that data to apply a certain logic, and then insert the data into Bigquery. I intend to use cloud scheduler, cloud pub/sub, cloud functions, and big query. Do not intend to use an apache beam as the volume of data is very low. (<100 records per day)
Complete writing cloud function which does the following:
1. Picks up data from pub/sub.
2. Processes the data and compares it to data in another table in BigQuery.
3. Based on certain conditions and criteria, the cloud function will insert that record into a separate table with an additional column that describes the status as 'Met', 'Not Met', 'Partially Met'
Please find project details in the attached excel file.
[login to view URL]
1. Data flow -> Pub/sub - Cloud function - Big Query
2. Flow logic - Every time data is pulled, the cloud function must use logic to map the ID, category, and status from the source table(highlighted in green) to Vul-CCF mapping, Vul-status to Control Health status, vul codename to app name with the environment and insert/update record in the destination table.
Bu iş için 6 freelancer ortalamada $71 teklif veriyor
Hi, I have a proven record for delivering quality websites to clients internationally, My reviews speaks for itself, I would like to offer you my services at a reasonable cost with a high end website for your business. Daha Fazla