I'm trying to automate the ingestion of data from Pub/sub to Bigquery. I need help writing a cloud function to subscribe to a pub/sub, do some processing on that data to apply a certain logic, and then insert the data into Bigquery. I intend to use cloud scheduler, cloud pub/sub, cloud functions, and big query. Do not intend to use an apache beam as the volume of data is very low. (<100 records per day)
Complete writing cloud function which does the following:
1. Picks up data from pub/sub.
2. Processes the data and compares it to data in another table in BigQuery.
3. Based on certain conditions and criteria, the cloud function will insert that record into a separate table with an additional column that describes the status as 'Met', 'Not Met', 'Partially Met'
Please find project details in the attached excel file.
[login to view URL]
1. Data flow -> Pub/sub - Cloud function - Big Query
2. Flow logic - Every time data is pulled, the cloud function must use logic to map the ID, category, and status from the source table(highlighted in green) to Vul-CCF mapping, Vul-status to Control Health status, vul codename to app name with the environment and insert/update record in the destination table.
Hi. I am an experienced Full stack developer with expertise in Python, GCP, Bigquery and many more. I have done a similar project in my past and am sure to complete this within reasonable time. In previous project , a Daha Fazla
Bu iş için 4 freelancer ortalamada $135 teklif veriyor
Hello,I work on GCP,AWS and Python having 14 years of professional experience,can create a cloud function using python as required,Thanks.
Hello there! Good day! I am interested in this project. I have experience working on similar projects for more than 8 years. I am an expert in Python/GCP. Please reach out to me to discuss further. Thank you!
AN EXPERT IN ABOVE NAMED SKILLS, DEAR CLIENT, Thank you for sharing your project requirements. After KEENLY and PROFESSIONALLY reading your description and being able to comprehend CLEARLY with Mentioned REQUIRED SKIL Daha Fazla