[Apr 12, 2025] Today Updated Professional-Data-Engineer Exam Dumps Actual Questions
Professional-Data-Engineer exam dumps with real Google questions and answers
NEW QUESTION # 79
Your software uses a simple JSON format for all messages. These messages are published to Google Cloud Pub/Sub, then processed with Google Cloud Dataflow to create a real-time dashboard for the CFO.
During testing, you notice that some messages are missing in the dashboard. You check the logs, and all messages are being published to Cloud Pub/Sub successfully. What should you do next?
- A. Run a fixed dataset through the Cloud Dataflow pipeline and analyze the output.
- B. Check the dashboard application to see if it is not displaying correctly.
- C. Switch Cloud Dataflow to pull messages from Cloud Pub/Sub instead of Cloud Pub/Sub pushing messages to Cloud Dataflow.
- D. Use Google Stackdriver Monitoring on Cloud Pub/Sub to find the missing messages.
Answer: A
NEW QUESTION # 80
You designed a database for patient records as a pilot project to cover a few hundred patients in three clinics. Your design used a single database table to represent all patients and their visits, and you used self-joins to generate reports. The server resource utilization was at 50%. Since then, the scope of the project has expanded. The database must now store 100 times more patient records. You can no longer run the reports, because they either take too long or they encounter errors with insufficient compute resources. How should you adjust the database design?
- A. Add capacity (memory and disk space) to the database server by the order of 200.
- B. Normalize the master patient-record table into the patient table and the visits table, and create other necessary tables to avoid self-join.
- C. Shard the tables into smaller ones based on date ranges, and only generate reports with prespecified date ranges.
- D. Partition the table into smaller tables, with one for each clinic. Run queries against the smaller table pairs, and use unions for consolidated reports.
Answer: B
NEW QUESTION # 81
What are the minimum permissions needed for a service account used with Google Dataproc?
- A. Read and write to Google Cloud Storage; write to Google Cloud Logging
- B. Execute to Google Cloud Storage; execute to Google Cloud Logging
- C. Write to Google Cloud Storage; read to Google Cloud Logging
- D. Execute to Google Cloud Storage; write to Google Cloud Logging
Answer: A
Explanation:
Service accounts authenticate applications running on your virtual machine instances to other Google Cloud Platform services. For example, if you write an application that reads and writes files on Google Cloud Storage, it must first authenticate to the Google Cloud Storage API. At a minimum, service accounts used with Cloud Dataproc need permissions to read and write to Google Cloud Storage, and to write to Google Cloud Logging.
Reference: https://cloud.google.com/dataproc/docs/concepts/service-accounts#important_notes
NEW QUESTION # 82
You are building a data pipeline on Google Cloud. You need to prepare data using a casual method for a
machine-learning process. You want to support a logistic regression model. You also need to monitor and
adjust for null values, which must remain real-valued and cannot be removed. What should you do?
- A. Use Cloud Dataprep to find null values in sample source data. Convert all nulls to 'none' using a Cloud
Dataproc job. - B. Use Cloud Dataprep to find null values in sample source data. Convert all nulls to 0 using a Cloud
Dataprep job. - C. Use Cloud Dataflow to find null values in sample source data. Convert all nulls to 'none' using a Cloud
Dataprep job. - D. Use Cloud Dataflow to find null values in sample source data. Convert all nulls to using a custom script.
Answer: C
NEW QUESTION # 83
You have enabled the free integration between Firebase Analytics and Google BigQuery. Firebase now automatically creates a new table daily in BigQuery in the format app_events_YYYYMMDD.You want to query all of the tables for the past 30 days in legacy SQL. What should you do?
- A. Use the TABLE_DATE_RANGEfunction
- B. Use SELECT IF.(date >= YYYY-MM-DD AND date <= YYYY-MM-DD
- C. Use the WHERE_PARTITIONTIMEpseudo column
- D. Use WHEREdate BETWEEN YYYY-MM-DD AND YYYY-MM-DD
Answer: A
Explanation:
Explanation/Reference:
Reference: https://cloud.google.com/blog/products/gcp/using-bigquery-and-firebase-analytics-to- understand-your-mobile-app?hl=am
NEW QUESTION # 84
For the best possible performance, what is the recommended zone for your Compute Engine instance and Cloud Bigtable instance?
- A. Have the Cloud Bigtable instance to be in the same zone as all of the consumers of your data.
- B. Have the Compute Engine instance in the furthest zone from the Cloud Bigtable instance.
- C. Have both the Compute Engine instance and the Cloud Bigtable instance to be in the same zone.
- D. Have both the Compute Engine instance and the Cloud Bigtable instance to be in different zones.
Answer: C
Explanation:
Explanation
It is recommended to create your Compute Engine instance in the same zone as your Cloud Bigtable instance for the best possible performance, If it's not possible to create a instance in the same zone, you should create your instance in another zone within the same region. For example, if your Cloud Bigtable instance is located in us-central1-b, you could create your instance in us-central1-f. This change may result in several milliseconds of additional latency for each Cloud Bigtable request.
It is recommended to avoid creating your Compute Engine instance in a different region from your Cloud Bigtable instance, which can add hundreds of milliseconds of latency to each Cloud Bigtable request.
Reference: https://cloud.google.com/bigtable/docs/creating-compute-instance
NEW QUESTION # 85
You are designing the database schema for a machine learning-based food ordering service that will predict what users want to eat. Here is some of the information you need to store:
The user profile: What the user likes and doesn't like to eat
The user account information: Name, address, preferred meal times
The order information: When orders are made, from where, to whom
The database will be used to store all the transactional data of the product. You want to optimize the data schema. Which Google Cloud Platform product should you use?
- A. Cloud Bigtable
- B. Cloud SQL
- C. BigQuery
- D. Cloud Datastore
Answer: C
NEW QUESTION # 86
Your neural network model is taking days to train. You want to increase the training speed. What can you do?
- A. Increase the number of input features to your model.
- B. Subsample your test dataset.
- C. Subsample your training dataset.
- D. Increase the number of layers in your neural network.
Answer: D
Explanation:
Reference: https://towardsdatascience.com/how-to-increase-the-accuracy-of-a-neural-network-9f5d1c6f407d
NEW QUESTION # 87
Flowlogistic Case Study
Company Overview
Flowlogistic is a leading logistics and supply chain provider. They help businesses throughout the world manage their resources and transport them to their final destination. The company has grown rapidly, expanding their offerings to include rail, truck, aircraft, and oceanic shipping.
Company Background
The company started as a regional trucking company, and then expanded into other logistics market.
Because they have not updated their infrastructure, managing and tracking orders and shipments has become a bottleneck. To improve operations, Flowlogistic developed proprietary technology for tracking shipments in real time at the parcel level. However, they are unable to deploy it because their technology stack, based on Apache Kafka, cannot support the processing volume. In addition, Flowlogistic wants to further analyze their orders and shipments to determine how best to deploy their resources.
Solution Concept
Flowlogistic wants to implement two concepts using the cloud:
Use their proprietary technology in a real-time inventory-tracking system that indicates the location of
their loads
Perform analytics on all their orders and shipment logs, which contain both structured and unstructured
data, to determine how best to deploy resources, which markets to expand info. They also want to use predictive analytics to learn earlier when a shipment will be delayed.
Existing Technical Environment
Flowlogistic architecture resides in a single data center:
Databases
- 8 physical servers in 2 clusters
- SQL Server - user data, inventory, static data
- 3 physical servers
- Cassandra - metadata, tracking messages
10 Kafka servers - tracking message aggregation and batch insert
Application servers - customer front end, middleware for order/customs
- 60 virtual machines across 20 physical servers
- Tomcat - Java services
- Nginx - static content
- Batch servers
Storage appliances
- iSCSI for virtual machine (VM) hosts
- Fibre Channel storage area network (FC SAN) - SQL server storage
Network-attached storage (NAS) image storage, logs, backups
10 Apache Hadoop /Spark servers
- Core Data Lake
- Data analysis workloads
20 miscellaneous servers
- Jenkins, monitoring, bastion hosts,
Business Requirements
Build a reliable and reproducible environment with scaled panty of production.
Aggregate data in a centralized Data Lake for analysis
Use historical data to perform predictive analytics on future shipments
Accurately track every shipment worldwide using proprietary technology
Improve business agility and speed of innovation through rapid provisioning of new resources
Analyze and optimize architecture for performance in the cloud
Migrate fully to the cloud if all other requirements are met
Technical Requirements
Handle both streaming and batch data
Migrate existing Hadoop workloads
Ensure architecture is scalable and elastic to meet the changing demands of the company.
Use managed services whenever possible
Encrypt data flight and at rest
Connect a VPN between the production data center and cloud environment
SEO Statement
We have grown so quickly that our inability to upgrade our infrastructure is really hampering further growth and efficiency. We are efficient at moving shipments around the world, but we are inefficient at moving data around.
We need to organize our information so we can more easily understand where our customers are and what they are shipping.
CTO Statement
IT has never been a priority for us, so as our data has grown, we have not invested enough in our technology. I have a good staff to manage IT, but they are so busy managing our infrastructure that I cannot get them to do the things that really matter, such as organizing our data, building the analytics, and figuring out how to implement the CFO' s tracking technology.
CFO Statement
Part of our competitive advantage is that we penalize ourselves for late shipments and deliveries. Knowing where out shipments are at all times has a direct correlation to our bottom line and profitability.
Additionally, I don't want to commit capital to building out a server environment.
Flowlogistic's management has determined that the current Apache Kafka servers cannot handle the data volume for their real-time inventory tracking system. You need to build a new system on Google Cloud Platform (GCP) that will feed the proprietary tracking software. The system must be able to ingest data from a variety of global sources, process and query in real-time, and store the data reliably. Which combination of GCP products should you choose?
- A. Cloud Load Balancing, Cloud Dataflow, and Cloud Storage
- B. Cloud Pub/Sub, Cloud SQL, and Cloud Storage
- C. Cloud Pub/Sub, Cloud Dataflow, and Cloud Storage
- D. Cloud Dataflow, Cloud SQL, and Cloud Storage
- E. Cloud Pub/Sub, Cloud Dataflow, and Local SSD
Answer: B
NEW QUESTION # 88
You are selecting services to write and transform JSON messages from Cloud Pub/Sub to BigQuery for a data pipeline on Google Cloud. You want to minimize service costs. You also want to monitor and accommodate input data volume that will vary in size with minimal manual intervention. What should you do?
- A. Use Cloud Dataflow to run your transformations. Monitor the job system lag with Stackdriver. Use the default autoscaling setting for worker instances.
- B. Use Cloud Dataproc to run your transformations. Monitor CPU utilization for the cluster. Resize the number of worker nodes in your cluster via the command line.
- C. Use Cloud Dataflow to run your transformations. Monitor the total execution time for a sampling of jobs.
Configure the job to use non-default Compute Engine machine types when needed. - D. Use Cloud Dataproc to run your transformations. Use the diagnose command to generate an operational output archive. Locate the bottleneck and adjust cluster resources.
Answer: A
Explanation:
Dataflow is good with autoscaling and stackdriver to monitor CPU and Storage.
NEW QUESTION # 89
Which of these statements about BigQuery caching is true?
- A. By default, a query's results are not cached.
- B. Query results are cached even if you specify a destination table.
- C. BigQuery caches query results for 48 hours.
- D. There is no charge for a query that retrieves its results from cache.
Answer: D
Explanation:
When query results are retrieved from a cached results table, you are not charged for the query.
BigQuery caches query results for 24 hours, not 48 hours.
Query results are not cached if you specify a destination table.
A query's results are always cached except under certain conditions, such as if you specify a destination table.
Reference: https://cloud.google.com/bigquery/querying-data#query-caching
NEW QUESTION # 90
You have enabled the free integration between Firebase Analytics and Google BigQuery. Firebase now
automatically creates a new table daily in BigQuery in the format app_events_YYYYMMDD. You want to
query all of the tables for the past 30 days in legacy SQL. What should you do?
- A. Use the TABLE_DATE_RANGE function
- B. Use SELECT IF.(date >= YYYY-MM-DD AND date <= YYYY-MM-DD
- C. Use the WHERE_PARTITIONTIME pseudo column
- D. Use WHERE date BETWEEN YYYY-MM-DD AND YYYY-MM-DD
Answer: A
NEW QUESTION # 91
Your organization has two Google Cloud projects, project A and project B. In project A, you have a Pub/Sub topic that receives data from confidential sources. Only the resources in project A should be able to access the data in that topic. You want to ensure that project B and any future project cannot access data in the project A topic. What should you do?
- A. Configure VPC Service Controls in the organization with a perimeter around project A.
- B. Use Identity and Access Management conditions to ensure that only users and service accounts in project A can access resources in project.
- C. Add firewall rules in project A so only traffic from the VPC in project A is permitted.
- D. Configure VPC Service Controls in the organization with a perimeter around the VPC of project A.
Answer: A
Explanation:
Identity and Access Management (IAM) is the recommended way to control access to Pub/Sub resources, such as topics and subscriptions. IAM allows you to grant roles and permissions to users and service accounts at the project level or the individual resource level. You can also use IAM conditions to specify additional attributes for granting or denying access, such as time, date, or origin. By using IAM conditions, you can ensure that only the resources in project A can access the data in the project A topic, regardless of the network configuration or the VPC Service Controls. You can also prevent project B and any future project from accessing the data in the project A topic by not granting them any roles or permissions on the topic.
Option A is not a good solution, as VPC Service Controls are designed to prevent data exfiltration from Google Cloud resources to the public internet, not to control access between Google Cloud projects. VPC Service Controls create a perimeter around the resources of one or more projects, and restrict the communication with resources outside the perimeter. However, VPC Service Controls do not apply to Pub/Sub, as Pub/Sub is not associated with any specific IP address or VPC network. Therefore, configuring VPC Service Controls with a perimeter around the VPC of project A would not prevent project B or any future project from accessing the data in the project A topic, if they have the necessary IAM roles and permissions.
Option B is not a good solution, as firewall rules are used to control the ingress and egress traffic to and from the VPC network of a project. Firewall rules do not apply to Pub/Sub, as Pub/Sub is not associated with any specific IP address or VPC network. Therefore, adding firewall rules in project A to only permit traffic from the VPC in project A would not prevent project B or any future project from accessing the data in the project A topic, if they have the necessary IAM roles and permissions.
Option C is not a good solution, as VPC Service Controls are designed to prevent data exfiltration from Google Cloud resources to the public internet, not to control access between Google Cloud projects. VPC Service Controls create a perimeter around the resources of one or more projects, and restrict the communication with resources outside the perimeter. However, VPC Service Controls do not apply to Pub/Sub, as Pub/Sub is not associated with any specific IP address or VPC network. Therefore, configuring VPC Service Controls with a perimeter around project A would not prevent project B or any future project from accessing the data in the project A topic, if they have the necessary IAM roles and permissions. Reference: Access control with IAM | Cloud Pub/Sub Documentation | Google Cloud, [Using IAM Conditions | Cloud IAM Documentation | Google Cloud], [VPC Service Controls overview | Google Cloud], [Using VPC Service Controls | Google Cloud], [Pub/Sub tier capabilities | Memorystore for Redis | Google Cloud].
NEW QUESTION # 92
You are on the data governance team and are implementing security requirements to deploy resources. You need to ensure that resources are limited to only the europe-west 3 region You want to follow Google-recommended practices What should you do?
- A. Create a Cloud Function to monitor all resources created and automatically destroy the ones created outside the europe-west3 region.
- B. Set the constraints/gcp. resourceLocations organization policy constraint to in: europe-west3-locations.
- C. Set the constraints/gcp. resourceLocations organization policy constraint to in:eu-locations.
- D. Deploy resources with Terraform and implement a variable validation rule to ensure that the region is set to the europe-west3 region for all resources.
Answer: B
Explanation:
To ensure that resources are limited to only the europe-west3 region, you should set the organization policy constraint constraints/gcp.resourceLocations to in:europe-west3-locations. This policy restricts the deployment of resources to the specified locations, which in this case is the europe-west3 region. By setting this policy, you enforce location compliance across your Google Cloud resources, aligning with the best practices for data governance and regulatory compliance.
References:
* Professional Data Engineer Certification Exam Guide | Learn - Google Cloud1.
* Preparing for Google Cloud Certification: Cloud Data Engineer2.
* Professional Data Engineer Certification | Learn | Google Cloud3.
3: Professional Data Engineer Certification | Learn | Google Cloud 2: Preparing for Google Cloud Certification: Cloud Data Engineer 1: Professional Data Engineer Certification Exam Guide | Learn - Google Cloud
NEW QUESTION # 93
You have several Spark jobs that run on a Cloud Dataproc cluster on a schedule. Some of the jobs run in sequence, and some of the jobs run concurrently. You need to automate this process. What should you do?
- A. Create a Cloud Dataproc Workflow Template
- B. Create an initialization action to execute the jobs
- C. Create a Bash script that uses the Cloud SDK to create a cluster, execute jobs, and then tear down the cluster
- D. Create a Directed Acyclic Graph in Cloud Composer
Answer: D
NEW QUESTION # 94
Which Cloud Dataflow / Beam feature should you use to aggregate data in an unbounded data source every hour based on the time when the data entered the pipeline?
- A. The with Allowed Lateness method
- B. An event time trigger
- C. A processing time trigger
- D. An hourly watermark
Answer: C
Explanation:
When collecting and grouping data into windows, Beam uses triggers to determine when to emit the aggregated results of each window.
Processing time triggers. These triggers operate on the processing time ?the time when the data element is processed at any given stage in the pipeline. Event time triggers. These triggers operate on the event time, as indicated by the timestamp on each data element. Beam's default trigger is event time-based.
Reference: https://beam.apache.org/documentation/programming-guide/#triggers
NEW QUESTION # 95
You operate a logistics company, and you want to improve event delivery reliability for vehicle-based sensors. You operate small data centers around the world to capture these events, but leased lines that provide connectivity from your event collection infrastructure to your event processing infrastructure are unreliable, with unpredictable latency. You want to address this issue in the most cost-effective way. What should you do?
- A. Establish a Cloud Interconnect between all remote data centers and Google.
- B. Have the data acquisition devices publish data to Cloud Pub/Sub.
- C. Write a Cloud Dataflow pipeline that aggregates all data in session windows.
- D. Deploy small Kafka clusters in your data centers to buffer events.
Answer: B
NEW QUESTION # 96
You have a requirement to insert minute-resolution data from 50,000 sensors into a BigQuery table. You expect significant growth in data volume and need the data to be available within 1 minute of ingestion for real-time analysis of aggregated trends. What should you do?
- A. Use a Cloud Dataflow pipeline to stream data into the BigQuery table.
- B. Use the MERGE statement to apply updates in batch every 60 seconds.
- C. Use bq load to load a batch of sensor data every 60 seconds.
- D. Use the INSERT statement to insert a batch of data every 60 seconds.
Answer: A
NEW QUESTION # 97
You are building a new application that you need to collect data from in a scalable way. Data arrives continuously from the application throughout the day, and you expect to generate approximately 150 GB of JSON data per day by the end of the year. Your requirements are:
* Decoupling producer from consumer
* Space and cost-efficient storage of the raw ingested data, which is to be stored indefinitely
* Near real-time SQL query
* Maintain at least 2 years of historical data, which will be queried with SQ Which pipeline should you use to meet these requirements?
- A. Create an application that writes to a Cloud SQL database to store the data. Set up periodic exports of the database to write to Cloud Storage and load into BigQuery.
- B. Create an application that publishes events to Cloud Pub/Sub, and create Spark jobs on Cloud Dataproc to convert the JSON data to Avro format, stored on HDFS on Persistent Disk.
- C. Create an application that provides an API. Write a tool to poll the API and write data to Cloud Storage as gzipped JSON files.
- D. Create an application that publishes events to Cloud Pub/Sub, and create a Cloud Dataflow pipeline that transforms the JSON event payloads to Avro, writing the data to Cloud Storage and BigQuery.
Answer: C
NEW QUESTION # 98
You are choosing a NoSQL database to handle telemetry data submitted from millions of Internet-of-Things (IoT) devices. The volume of data is growing at 100 TB per year, and each data entry has about 100 attributes.
The data processing pipeline does not require atomicity, consistency, isolation, and durability (ACID). However, high availability and low latency are required.
You need to analyze the data by querying against individual fields. Which three databases meet your requirements? (Choose three.)
- A. HDFS with Hive
- B. Redis
- C. MongoDB
- D. MySQL
- E. Cassandra
- F. HBase
Answer: A,C,F
NEW QUESTION # 99
An organization maintains a Google BigQuery dataset that contains tables with user-level data. They want to expose aggregates of this data to other Google Cloud projects, while still controlling access to the user-level data. Additionally, they need to minimize their overall storage cost and ensure the analysis cost for other projects is assigned to those projects. What should they do?
- A. Create dataViewer Identity and Access Management (IAM) roles on the dataset to enable sharing.
- B. Create and share a new dataset and view that provides the aggregate results.
- C. Create and share an authorized view that provides the aggregate results.
- D. Create and share a new dataset and table that contains the aggregate results.
Answer: A
NEW QUESTION # 100
You have a BigQuery dataset named "customers". All tables will be tagged by using a Data Catalog tag template named "gdpr". The template contains one mandatory field, "has sensitive data~. with a boolean value. All employees must be able to do a simple search and find tables in the dataset that have either true or false in the "has sensitive data" field. However, only the Human Resources (HR) group should be able to see the data inside the tables for which "hass-ensitive-data" is true. You give the all employees group the bigquery.metadataViewer and bigquery.connectionUser roles on the dataset. You want to minimize configuration overhead. What should you do next?
- A. Create the ~gdpr" tag template with private visibility. Assign the datacatalog. tagTemplateViewer role on this tag to the all employees group, and assign the bigquery.dataViewer role to the HR group on the tables that contain sensitive data.
- B. Create the "gdpr" tag template with public visibility. Assign the bigquery. dataViewer role to the HR group on the tables that contain sensitive data.
- C. Create the "gdpr" tag template with private visibility. Assign the bigquery -dataViewer role to the HR group on the tables that contain sensitive data.
- D. Create the "gdpr" tag template with public visibility. Assign the datacatalog. tagTemplateViewer role on this tag to the all employees.group, and assign the bijquery.dataViewer role to the HR group on the tables that contain sensitive data.
Answer: D
Explanation:
To ensure that all employees can search and find tables with GDPR tags while restricting data access to sensitive tables only to the HR group, follow these steps:
Data Catalog Tag Template:
Use Data Catalog to create a tag template named "gdpr" with a boolean field "has sensitive data". Set the visibility to public so all employees can see the tags.
Roles and Permissions:
Assign the datacatalog.tagTemplateViewer role to the all employees group. This role allows users to view the tags and search for tables based on the "has sensitive data" field.
Assign the bigquery.dataViewer role to the HR group specifically on tables that contain sensitive data. This ensures only HR can access the actual data in these tables.
Steps to Implement:
Create the GDPR Tag Template:
Define the tag template in Data Catalog with the necessary fields and set visibility to public.
Assign Roles:
Grant the datacatalog.tagTemplateViewer role to the all employees group for visibility into the tags.
Grant the bigquery.dataViewer role to the HR group on tables marked as having sensitive data.
Reference:
Data Catalog Documentation
Managing Access Control in BigQuery
IAM Roles in Data Catalog
NEW QUESTION # 101
Your company's customer_order table in BigOuery stores the order history for 10 million customers, with a table size of 10 PB. You need to create a dashboard for the support team to view the order history. The dashboard has two filters, countryname and username. Both are string data types in the BigQuery table. When a filter is applied, the dashboard fetches the order history from the table and displays the query results. However, the dashboard is slow to show the results when applying the filters to the following query:
How should you redesign the BigQuery table to support faster access?
- A. Partition the table by country and username fields.
- B. Cluster the table by country field, and partition by username field.
- C. Partition the table by _PARTITIONTIME.
- D. Cluster the table by country and username fields
Answer: D
Explanation:
To improve the performance of querying a large BigQuery table with filters on countryname and username, clustering the table by these fields is the most effective approach. Here's why option C is the best choice:
Clustering in BigQuery:
Clustering organizes data based on the values in specified columns. This can significantly improve query performance by reducing the amount of data scanned during query execution.
Clustering by countryname and username means that data is physically sorted and stored together based on these fields, allowing BigQuery to quickly locate and read only the relevant data for queries using these filters.
Filter Efficiency:
With the table clustered by countryname and username, queries that filter on these columns can benefit from efficient data retrieval, reducing the amount of data processed and speeding up query execution.
This directly addresses the performance issue of the dashboard queries that apply filters on these fields.
Steps to Implement:
Redesign the Table:
Create a new table with clustering on countryname and username:
CREATE TABLE project.dataset.new_table
CLUSTER BY countryname, username AS
SELECT * FROM project.dataset.customer_order;
Migrate Data:
Transfer the existing data from the original table to the new clustered table.
Update Queries:
Modify the dashboard queries to reference the new clustered table.
Reference:
BigQuery Clustering Documentation
Optimizing Query Performance
NEW QUESTION # 102
MJTelco Case Study
Company Overview
MJTelco is a startup that plans to build networks in rapidly growing, underserved markets around the world. The company has patents for innovative optical communications hardware. Based on these patents, they can create many reliable, high-speed backbone links with inexpensive hardware.
Company Background
Founded by experienced telecom executives, MJTelco uses technologies originally developed to overcome communications challenges in space. Fundamental to their operation, they need to create a distributed data infrastructure that drives real-time analysis and incorporates machine learning to continuously optimize their topologies. Because their hardware is inexpensive, they plan to overdeploy the network allowing them to account for the impact of dynamic regional politics on location availability and cost.
Their management and operations teams are situated all around the globe creating many-to-many relationship between data consumers and provides in their system. After careful consideration, they decided public cloud is the perfect environment to support their needs.
Solution Concept
MJTelco is running a successful proof-of-concept (PoC) project in its labs. They have two primary needs:
* Scale and harden their PoC to support significantly more data flows generated when they ramp to more than
50,000 installations.
* Refine their machine-learning cycles to verify and improve the dynamic models they use to control topology definition.
MJTelco will also use three separate operating environments - development/test, staging, and production - to meet the needs of running experiments, deploying new features, and serving production customers.
Business Requirements
* Scale up their production environment with minimal cost, instantiating resources when and where needed in an unpredictable, distributed telecom user community.
* Ensure security of their proprietary data to protect their leading-edge machine learning and analysis.
* Provide reliable and timely access to data for analysis from distributed research workers
* Maintain isolated environments that support rapid iteration of their machine-learning models without affecting their customers.
Technical Requirements
Ensure secure and efficient transport and storage of telemetry data
Rapidly scale instances to support between 10,000 and 100,000 data providers with multiple flows each.
Allow analysis and presentation against data tables tracking up to 2 years of data storing approximately 100m records/day Support rapid iteration of monitoring infrastructure focused on awareness of data pipeline problems both in telemetry flows and in production learning cycles.
CEO Statement
Our business model relies on our patents, analytics and dynamic machine learning. Our inexpensive hardware is organized to be highly reliable, which gives us cost advantages. We need to quickly stabilize our large distributed data pipelines to meet our reliability and capacity commitments.
CTO Statement
Our public cloud services must operate as advertised. We need resources that scale and keep our data secure.
We also need environments in which our data scientists can carefully study and quickly adapt our models.
Because we rely on automation to process our data, we also need our development and test environments to work as we iterate.
CFO Statement
The project is too large for us to maintain the hardware and software required for the data and analysis. Also, we cannot afford to staff an operations team to monitor so many data feeds, so we will rely on automation and infrastructure. Google Cloud's machine learning will allow our quantitative researchers to work on our high- value problems instead of problems with our data pipelines.
You need to compose visualization for operations teams with the following requirements:
* Telemetry must include data from all 50,000 installations for the most recent 6 weeks (sampling once every minute)
* The report must not be more than 3 hours delayed from live data.
* The actionable report should only show suboptimal links.
* Most suboptimal links should be sorted to the top.
* Suboptimal links can be grouped and filtered by regional geography.
* User response time to load the report must be <5 seconds.
You create a data source to store the last 6 weeks of data, and create visualizations that allow viewers to see multiple date ranges, distinct geographic regions, and unique installation types. You always show the latest data without any changes to your visualizations. You want to avoid creating and updating new visualizations each month. What should you do?
- A. Load the data into relational database tables, write a Google App Engine application that queries all rows, summarizes the data across each criteria, and then renders results using the Google Charts and visualization API.
- B. Export the data to a spreadsheet, compose a series of charts and tables, one for each possible combination of criteria, and spread them across multiple tabs.
- C. Look through the current data and compose a series of charts and tables, one for each possible combination of criteria.
- D. Look through the current data and compose a small set of generalized charts and tables bound to criteria filters that allow value selection.
Answer: D
NEW QUESTION # 103
Your company uses a proprietary system to send inventory data every 6 hours to a data ingestion service in the cloud. Transmitted data includes a payload of several fields and the timestamp of the transmission. If there are any concerns about a transmission, the system re-transmits the dat
a. How should you deduplicate the data most efficiency?
- A. Assign global unique identifiers (GUID) to each data entry.
- B. Maintain a database table to store the hash value and other metadata for each data entry.
- C. Compute the hash value of each data entry, and compare it with all historical data.
- D. Store each data entry as the primary key in a separate database and apply an index.
Answer: B
NEW QUESTION # 104
......
Exam Sure Pass Google Certification with Professional-Data-Engineer exam questions: https://www.actual4cert.com/Professional-Data-Engineer-real-questions.html
Professional-Data-Engineer Exam in First Attempt Guaranteed: https://drive.google.com/open?id=1-J0xPVDAMuedTmtrA37aRApF6ZzFm4B2