Table of Contents
In this article, we will see how to Install Google Cloud BigQuery Python client library in python environment on Linux. If you are looking to interact with Google Cloud BigQuery service using python programming language then you are going to need google-cloud-bigquery client library. This library helps you interact with BigQuery programmatically. You can easily manage bigquery datasets, tables and jobs using this client library.
You can also execute SQL queries on datasets stored in BigQuery using help of this client library. If you are looking to load data from multiple different sources like csv, json or cloud storage then you can make use of this library. Similarly there are so many other tasks that you can perform in BigQuery by using this client library.
How to Install Google Cloud BigQuery Python client library on Linux
Also Read: How to Install Elasticvue desktop on Ubuntu Linux
Step 1: Prerequisites
a) You should have a running Linux Server.
b) You should have Python 3.7 or higher installed in your System.
c) You should have pip utility installed in your System.
d) Minimum hardware requirements :-
- CPU: Any CPU with 64-bit processor would be enough.
- Memory: A minimum of 512MB is required but 2GB would be highly recommended for greater performance.
- Storage: A minimum of 100MB would be required for client library and its dependencies installation.
Step 2: Install BigQuery Python client library
In the next step, you can install bigquery python client library by using pip install google-cloud-bigquery command as shown below.
cyberithub@ubuntu:~$ pip install google-cloud-bigquery
Collecting google-cloud-bigquery
Downloading google_cloud_bigquery-3.27.0-py2.py3-none-any.whl (240 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 240.1/240.1 KB 2.1 MB/s eta 0:00:00
Collecting google-auth<3.0.0dev,>=2.14.1
Downloading google_auth-2.36.0-py2.py3-none-any.whl (209 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.5/209.5 KB 1.4 MB/s eta 0:00:00
Collecting packaging>=20.0.0
Downloading packaging-24.2-py3-none-any.whl (65 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.5/65.5 KB 3.0 MB/s eta 0:00:00
Collecting google-cloud-core<3.0.0dev,>=2.4.1
Downloading google_cloud_core-2.4.1-py2.py3-none-any.whl (29 kB)
Requirement already satisfied: python-dateutil<3.0dev,>=2.7.3 in /usr/lib/python3/dist-packages (from google-cloud-bigquery) (2.8.1)
Requirement already satisfied: requests<3.0.0dev,>=2.21.0 in /usr/lib/python3/dist-packages (from google-cloud-bigquery) (2.25.1)
Collecting google-api-core[grpc]<3.0.0dev,>=2.11.1
Downloading google_api_core-2.23.0-py3-none-any.whl (156 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 156.6/156.6 KB 4.8 MB/s eta 0:00:00
Collecting google-resumable-media<3.0dev,>=2.0.0
Downloading google_resumable_media-2.7.2-py2.py3-none-any.whl (81 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 81.3/81.3 KB 4.4 MB/s eta 0:00:00
.............................................
Step 3: Verify Installation
To verify installation of client library in your python environment, run pip show google-cloud-bigquery command. If it is already installed then you should see an output like below.
cyberithub@ubuntu:~$ pip show google-cloud-bigquery Name: google-cloud-bigquery Version: 3.27.0 Summary: Google BigQuery API client library Home-page: https://github.com/googleapis/python-bigquery Author: Google LLC Author-email: googleapis-packages@google.com License: Apache 2.0 Location: /home/cyberithub/.local/lib/python3.10/site-packages Requires: google-api-core, google-auth, google-cloud-core, google-resumable-media, packaging, python-dateutil, requests Required-by:
Step 4: Using BigQuery Python client library
Now that client library is installed, let's see how to use this library in python to interact with Google Cloud BigQuery service. For our demo purpose, we have below vehicle table available:-
| car | color |
|---|---|
| Ford | red |
| Toyota | white |
We can query car name and color from above table using below python script:-
from google.cloud import bigquery
client = bigquery.Client()
query = "SELECT car, color FROM `cyberithub.sample.vehicle`"
query_job = client.query(query)
for row in query_job.result():
print(f"car: {row.car}, color: {row.color}")
Let's understand different components of above example. In the first, we are importing bigquery client library to enable python to interact with Google Cloud BigQuery service.
from google.cloud import bigquery
Then we are initializing a bigquery client instance using our google cloud credentials. In case if you would like set up authentication credentials through service account key then you can follow steps mentioned on How to Add a Service Accounts Key in Google Cloud in 7 Easy Steps article. Please remember without proper authentication, client instance won't get created.
client = bigquery.Client()
Next we are defining our query to get the data from vehicle table in sample dataset under cyberithub project in Google Cloud. In our query, we are simply fetching car and color from vehicle table.
query = "SELECT car, color FROM `cyberithub.sample.vehicle`"
Then we are submitting above query to get the data from BigQuery. Below code will return a QueryJob object.
query_job = client.query(query)
Finally, to display the result, we are using for loop to iterate over number of rows received and displaying car and color values from each row on the output.
for row in query_job.result(): print(f"car: {row.car}, color: {row.color}")
If you run above python script, you should see an output like below:-
car: Ford, color: red
car: Toyota, color: white
This confirms client library in python is working as expected.
Step 5: Uninstall BigQuery Python client library
If you are done using Bigquery python client library then you can also choose to remove it from your python environment using pip uninstall google-cloud-bigquery command as shown below.
cyberithub@ubuntu:~$ pip uninstall google-cloud-bigquery Found existing installation: google-cloud-bigquery 3.27.0 Uninstalling google-cloud-bigquery-3.27.0: Would remove: /usr/local/lib/python3.10/dist-packages/google/cloud/bigquery/* /usr/local/lib/python3.10/dist-packages/google/cloud/bigquery_v2/* /usr/local/lib/python3.10/dist-packages/google_cloud_bigquery-3.27.0.dist-info/* Proceed (Y/n)? Y Successfully uninstalled google-cloud-bigquery-3.27.0