
Published 2023-02-15 21:23:13
How to Read and Write JSON Files in Python
thanks to freepik for wonderful images
#pythonlearning #python #pythonprogramming #programming
TL;DR
This article explains what JSON is and how to work with it in Python. It covers the data types that can be converted to and from JSON, the Python json module, serialization and deserialization, reading JSON from a file, performing changes to JSON, and working with API calls using the requests library and JSON.
Source Code
You can find the source code under the following link:
https://github.com/devoriales/python/tree/main/json
Introduction To JSON
JSON is a lightweight data interchange format that has become increasingly popular in recent years due to its simplicity and ease of use.
It is widely used for data exchange between applications, especially microservices , as it is both human-readable and machine-readable.
It is natively supported in Python through the json
module, which is included in the standard library of the Python programming language. This means that no additional installation is required to use the json
module, and it can be imported and used in Python code without any external dependencies. The json
module provides a set of methods for encoding and decoding JSON data, as well as reading and writing JSON files.
We will cover the basics of JSON, how to read and write JSON files in Python, and how to work with JSON data using Python's built-in libraries.
This post will provide you with an understanding of how to work with JSON data using Python.
How is JSON data made up?
JSON data is made up of key-value pairs, similar to a dictionary in Python or an object in JavaScript. The keys are always strings, while the values can be strings, numbers, booleans, arrays, or even other JSON objects.
❗While JSON and Python dictionaries are similar, they are not exactly the same. JSON data is represented as a string and conforms to a specific syntax that defines objects, arrays, strings, numbers, and booleans. Python dictionaries are a built-in data type in Python that represent a collection of key-value pairs.
Python dictionaries are not limited to a specific syntax, and can contain keys and values of any valid Python data type, including nested dictionaries, lists, and more.
JSON objects are enclosed in curly braces { }
and key-value pairs are separated by a colon :
Arrays, which are a collection of values, are enclosed in square brackets [ ]
and values are separated by commas.
Here's an example of a simple JSON object:
{
"manufacturer": "Ferrari",
"model": "LaFerrari",
"year": 2013,
"engine": {
"type": "hybrid",
"displacement": 6.3
},
"topSpeed": 217,
"acceleration": {
"zeroToSixty": 2.4,
"zeroToTwoHundred": 15
},
"colors": ["rosso corsa", "yellow", "black"],
"isLimitedEdition": true
}
In this example, the JSON object represents a Ferrari LaFerrari, a hybrid super sports car, introduced in 2013.
This JSON object has key-value pairs which describes different aspects of the car.
The json
library in Python provides methods to encode Python objects into JSON format and decode JSON data into Python objects.
What data types can we convert between Python and JSON?
Python dictionaries and JSON share many data types, allowing for easy conversion between the two formats. However, there are some differences to consider when working with them.
One key difference is that JSON data is always represented as a string, whereas Python dictionaries are a built-in data type in Python. Additionally, JSON objects can only have string keys, while Python dictionaries can have keys of any valid Python data type.
Another difference is that JSON objects have a limited depth of nesting, while Python dictionaries can be nested to any level. Additionally, while Python dictionaries can include comments using the # symbol, JSON does not support comments.
Finally, it's worth noting that JSON is based on a subset of the JavaScript programming language, while Python is a separate programming language. Additionally, JSON supports a limited set of data types, whereas Python dictionaries can contain any valid Python data type.
Here are the data types that can be converted between Python and JSON:
Python | JSON | Details |
---|---|---|
dict | Object | Python dictionaries can be converted to JSON objects and vice versa. Keys in the dictionary are converted to JSON strings and values can be any valid JSON value. |
list | Array | Python lists can be converted to JSON arrays and vice versa. Items in the list can be any valid JSON value. |
tuple | Array | |
str | String | Python integers and floats can be converted to JSON numbers and vice versa. |
int | Number | Python integers and floats can be converted to JSON numbers and vice versa. |
float | Number | Python integers and floats can be converted to JSON numbers and vice versa. |
True | true | Python booleans (True and False ) can be converted to JSON booleans and vice versa. |
False | false | |
None | null |
What methods are provided by the json
module?
The json
module in Python provides a set of methods for working with JSON data. Here are the main methods provided by the json
module:
json.loads()
: converts a JSON string into a Python object.json.load()
: reads a JSON file and convert its contents into a Python object.json.dumps()
: converts a Python object into a JSON string.json.dump()
: converts a Python object to a JSON file.
The following are some additional submemethods that you can use for serializing and deserializing the JSON data:
skipkeys
: accepts a boolean flag to determine whether keys that are not strings should be skipped.ensure_ascii
: accepts a boolean flag to determine whether non-ASCII characters should be escaped during serialization.check_circular
: accepts a boolean flag (True/False) to decide whether circular references should be checked during serialization.allow_nan
: accepts a boolean flag (True/False) to determine whether NaN, Infinity, and -Infinity values should be allowed during serialization.indent
: sets the indentation level for the serialized JSON string.separators
: determines how commas and colons should be used in the serialized JSON string.sort_keys
: accepts a boolean flag (True/False) that you can set to determine whether the keys in the serialized JSON string should be sorted.
Overall, the json
module provides a range of methods and submethods that make it easy to work with JSON data in Python, and allow for a high degree of customization in how the data is serialized and deserialized.
Serialization and Deserialization
We need to learn some concept that are pretty common in data interchange and communication, specially while exchanging data between different systems or programming languages.
Serialization is the process of converting a data structure into a format that can be transmitted or stored, such as a string or binary format. In Python, the json
module provides methods for serializing Python objects (such as dictionaries or lists) into JSON strings, which can be easily transmitted and stored in a variety of contexts.
Deserialization is the opposite process which is converting JSON data into a Python object. Deserialization is commonly used when receiving JSON data from an external source, such as a web API or a file.
In the next following sections, we will learn how to serialize and deserialize data with help of the json module.
Deserialize Data
In the following example, we will be deserializing the data from a JSON string to a Python dictionary.
To work with the JSON object we defined earlier in Python, we can first deserialize the JSON data into a Python object using the json.loads()
method. The json.loads()
method takes a JSON string as its argument, and returns a Python object that represents the deserialized data.
Example:
import json
# JSON data as a string
json_data = '{"manufacturer": "Ferrari", "model": "LaFerrari", "year": 2013, "engine": {"type": "hybrid", "displacement": 6.3}, "topSpeed": 217, "acceleration": {"zeroToSixty": 2.4, "zeroToTwoHundred": 15}, "colors": ["rosso corsa", "yellow", "black"], "isLimitedEdition": true}'
# Load the JSON data into a Python object
car_data = json.loads(json_data)
# Print the Python object
print(car_data)
Output:
{
"manufacturer": "Ferrari",
"model": "LaFerrari",
"year": 2013,
"engine": {
"type": "hybrid",
"displacement": 6.3
},
"topSpeed": 217,
"acceleration": {
"zeroToSixty": 2.4,
"zeroToTwoHundred": 15
},
"colors": ["rosso corsa", "yellow", "black"],
"isLimitedEdition": true
}
Once we have loaded the JSON data into a Python object, we can access the individual values using Python's dictionary syntax. For example, to access the manufacturer of the car, we can use car_data["manufacturer"]
. Similarly, to access the acceleration performance, we can use car_data["acceleration"]["zeroToSixty"]
.
# print individual values
print(car_data["manufacturer"])
print(car_data["model"])
print(car_data["acceleration"]["zeroToSixty"])
Output:
Ferrari
LaFerrari
2.4
❗Please note, you wouldn't be able to access the data with a JSON string.
Serialize Data
In the following example, we will serialize data, which means converting data from a Python dictionary to a JSON string. Serialization is the process of converting a Python object into a JSON string that can be easily transmitted or stored in a variety of contexts:
# Convert the Python object to a JSON string
json_string = json.dumps(car_data)
# Print the JSON string
print(json_string)
Output:
{"manufacturer": "Ferrari", "model": "LaFerrari", "year": 2013, "engine": {"type": "hybrid", "displacement": 6.3}, "topSpeed": 217, "acceleration": {"zeroToSixty": 2.4, "zeroToTwoHundred": 15}, "colors": ["rosso corsa", "yellow", "black"], "isLimitedEdition": true}
Let's print out the types of the objects we are working with:
print(type(json_data))
print(type(car_data))
Output:
<class 'str'>
<class 'dict'>
As we can see, the json_data
was a string, and we used the json.loads()
method to convert it to a Python dictionary object, which is represented by the car_data
variable. Therefore, the output shows that json_data
is of type str
, while car_data
is of type dict
.
Read JSON data from a file
Use Case:
We run a car dealership, and a potential buyer has asked us to provide an offer for two sports cars. We recently requested an offer from our supplier for these two cars, and the supplier has sent us a JSON document that contains information about both cars, including all available options for each car.
To work with this data, we need to extract the information from the JSON file and display it in a clear and organized way. This involves reading the JSON file using Python, parsing the data into a
Python object, and accessing the relevant properties to extract the information we need. The data provided is quite large and can be found in the Git repository linked at the beginning of this article.
The directory structure looks like this:
├── data
│ └── cars <<<<<< the data file
├── get_data_from_files.py <<<< file containing the python code
Excerpt from the file:
{
"cars": [
{
"manufacturer": "Audi",
"model": "R8",
"year": 2022,
"engine": {
"type": "V10",
"displacement": 5.2,
"horsepower": 602,
"torque": 413
},
"transmission": {
"type": "automatic
...
In our Python code, we will write a script to convert the JSON data into a Python dictionary:
'''
devoriales.com, 2023
Path: json/get_data_from_files.py
description: get data from files
'''
import json
# get data from a file in data folder
with open('data/cars') as f: # open the file in read mode
data = json.load(f) # type dict
# print the data
print(data)
{'cars': [{'manufacturer': 'Audi', 'model': 'R8', 'year': 2022, 'engine': {'type': 'V10', 'displacement': 5.2, 'horsepower': 602, 'torque': 413}, 'transmission': {'type': 'automatic', 'speeds': 7, 'manufacturer': 'ZF', 'model': '8HP'}, 'drive': 'all-wheel', 'fuelType': 'gasoline', 'fuelEconomy': {'city': 13, 'highway': 20, 'combined': 16}, 'colors': [{'name': 'Mythos Black Metallic', 'code': '#212121'}, {'name': 'Tango Red Metallic', 'code': '#a01916'}, {'name': 'Daytona Gray Pearl Effect', 'code': '#6c757d'}], 'features': ['Bang & Olufsen sound system', 'Virtual cockpit', 'Full LED headlights', 'Carbon-ceramic brakes', 'Sport exhaust system'], 'options': {'interior': [{'name': 'Black', 'price': 0}, {'name': 'Express Red', 'price': 750}], 'wheels': [{'name': '19" 5-arm-spoke design', 'price': 0}, {'name': '20" 10-spoke-Y design', 'price': 1500}]}, 'price': {'base': 142700, 'options': 2250, 'delivery': 1250, 'total': 146200}}, {'manufacturer': 'Porsche', 'model': '911 Carrera S', 'year': 2022, 'engine': {'type': 'flat-six', 'displacement': 3.0, 'horsepower': 443, 'torque': 390}, 'transmission': {'type': 'automatic', 'speeds': 8, 'manufacturer': 'PDK'}, 'drive': 'rear-wheel', 'fuelType': 'gasoline', 'fuelEconomy': {'city': 18, 'highway': 24, 'combined': 20}, 'colors': [{'name': 'Jet Black Metallic', 'code': '#0c0c0d'}, {'name': 'Crayon', 'code': '#c3c1c1'}, {'name': 'Carmine Red', 'code': '#8c0404'}], 'features': ['Sport Chrono package', 'Porsche Active Suspension Management (PASM)', 'LED headlights with Porsche Dynamic Light System Plus (PDLS+)', 'Bose surround sound system', 'Sport exhaust system'], 'options': {'interior': [{'name': 'Black', 'price': 0}, {'name': 'Slate Grey', 'price': 4300}], 'wheels': [{'name': '20"/21" Carrera S', 'price': 0}, {'name': '20"/21" RS Spyder Design', 'price': 3270}], 'performance': [{'name': 'Porsche Ceramic Composite Brakes (PCCB)', 'price': 8400}, {'name': 'Front-axle lift system', 'price': 2600}]}, 'price': {'base': 121000, 'options': 20220, 'delivery': 1350, 'total': 143570}}]}
The data structure we're working with is quite complex. Let's begin with a simple task of printing out the manufacturer of each car:
# print the manufacturer of all the cars
for car in data['cars']:
print(car['manufacturer'])
Output:
Audi
Porsche
We're also interested in each car's model, so we'll modify our code to include this information as well:
# print the manufacturer of all the cars
for car in data['cars']:
print(car['manufacturer'], end=': ')
print(car['model'])
Output:
Audi: R8
Porsche: 911 Carrera S
Work with existing data in JSON
Now it's time to learn how to manipulate the data we've obtained in JSON format.
For this use case, we'll be using the same cars file, which includes data for two models. To modify the JSON data, we'll need to follow this pattern:
- Get Data: JSON data
- Deserialisation: convert JSON string to Python Dictionary with
json.load()
- Perform changes in Python dictionary
- Serialisation: convert Python dictionary to JSON string with
json.dump()
❗When working with JSON data stored in a file, we need to use the json.load()
method to read the data from the file and deserialize it into a Python object. This method takes a file object as its argument and returns a Python object such as a dictionary or a list that represents the data in the file.
Once we have made any necessary modifications to the Python object, we can use the json.dump()
method to serialize the data back into the JSON format and write it back to the file. This method takes two arguments: the Python object to serialize and a file object to write the serialized data to.
By using these two methods, we can read JSON data from a file, manipulate it in a Python script, and then write the modified data back to the file in JSON format.
Get Data
We can begin by reading the cars
data file:
'''
devoriales.com
Path: json/learn_json_1.py
description: json
JSON stands for JavaScript Object Notation
'''
import json
# JSON data as a string
with open('data/cars') as f:
# turn the JSON data into a Python object
data = json.load(f)
print(data)
Output:
{'cars': [{'manufacturer': 'Audi', 'model': 'R8', 'year': 2022, 'topSpeed': 205, 'engine': {'type': 'V10', 'displacement': 5.2, 'horsepower': 602, 'torque': 413}, 'transmission': {'type': 'automatic', 'speeds': 7, 'manufacturer': 'ZF', 'model': '8HP'}, 'drive': 'all-wheel', 'fuelType': 'gasoline', 'fuelEconomy': {'city': 13, 'highway': 20, 'combined': 16}, 'colors': [{'name': 'Mythos Black Metallic', 'code': '#212121'}, {'name': 'Tango Red Metallic', 'code': '#a01916'}, {'name': 'Daytona Gray Pearl Effect', 'code': '#6c757d'}], 'features': ['Bang & Olufsen sound system', 'Virtual cockpit', 'Full LED headlights', 'Carbon-ceramic brakes', 'Sport exhaust system'],
...
Deserialisation of the data
The formatting of the previous output is not the best since it's extremely hard to read the data. To make it more JSON-like, we can convert it to a JSON string and use the indent
parameter to add indentation:
'''
devoriales.com
Path: json/learn_json_1.py
description: json
JSON stands for JavaScript Object Notation
'''
import json
# JSON data as a string
with open('data/cars') as f:
# turn the JSON data into a Python object
data = json.load(f)
json_string = json.dumps(data, indent=4)
print(json_string)
Output:
{
"cars": [
{
"manufacturer": "Audi",
"model": "R8",
"year": 2022,
"topSpeed": 205,
"engine": {
"type": "V10",
"displacement": 5.2,
"horsepower": 602,
"torque": 413
},
...
To achieve this format, we used the dumps()
method, which takes two arguments: the Python object to serialize and an optional indent
parameter that specifies the number of spaces to use for indentation in the resulting JSON string.
However, one of the challenges of storing JSON data as a string is that it may not be as easy to manipulate the data as it would be when stored as a Python dictionary. Since a JSON string is simply a sequence of characters, manipulating the data requires additional string manipulation operations, which can be more complex than working with a dictionary directly.
Since we've already created the dictionary using load()
, we can use the dictionary stored in the data
variable to manipulate the key-value pairs that we want to change. Additionally, we can also add more data or remove data that we don't need.
Let's say that we want to make this information more simplified and relevant to our business.
For instance we don't need displacement and torque properties. part of the nested object called engine
Perform Changes
In this section, we'll learn how to modify Python dictionary data and delete unwanted entries.
Delete keys in Python dictionary
There are two primary methods for removing key-value pairs from a Python dictionary:
- The
pop()
method - The
del()
method
Delete with pop()
We will delete the "displacement
" and "torque
" keys in the data dictionary that we have already created.
The following code uses the pop()
method to remove the "displacement
" and "torque
" keys from the "engine" dictionary:
# remove displacement and torque from the data
for car in data['cars']:
car['engine'].pop('displacement')
car['engine'].pop('torque')
print(json.dumps(data, indent=2))
Output - modified data:
{
"cars": [
{
"manufacturer": "Audi",
"model": "R8",
"year": 2022,
"topSpeed": 205,
"engine": {
"type": "V10",
"horsepower": 602
}
},
{
"manufacturer": "Porsche",
"model": "911",
"year": 2021,
"topSpeed": 191,
"engine": {
"type": "flat-6",
"horsepower": 502
}
}
]
}
Delete with del()
In this section, we will delete the "manufacturer" key from the "transmission" object of each "car" object in the "cars" array using the del
method:
{
"manufacturer": "Porsche",
"model": "911 Carrera S",
"year": 2022,
"topSpeed": 192,
"engine": {
"type": "flat-six",
"displacement": 3.0,
"horsepower": 443,
"torque": 390
},
"transmission": {
"type": "automatic",
"speeds": 8,
"manufacturer": "PDK"
},
Delete the delete the manufacturer of the transmission:
# delete manufacturer of the transmission from the data with del()
for car in data['cars']:
del (car['transmission']['manufacturer'])
print(json.dumps(data, indent=2))
Output (partial):
{
"manufacturer": "Porsche",
"model": "911 Carrera S",
"year": 2022,
"topSpeed": 192,
"engine": {
"type": "flat-six",
"displacement": 3.0,
"horsepower": 443,
"torque": 390
},
"transmission": {
"type": "automatic",
"speeds": 8
},
...
What is the difference between pop() and del()
Both the pop()
method and the del()
statement are used to remove elements from a dictionary, but they have some important differences:
pop()
removes a key-value pair from the dictionary and returns the corresponding value. If the specified key is not found,pop()
raises aKeyError
. If a default value is provided as an argument,pop()
will return that value instead of raising an error.
The following code shows the difference between del()
and pop()
:
# create a dictionary of car objects
cars = {
"BMW": {"year": 2022, "model": "M4", "color": "red"},
"Tesla": {"year": 2021, "model": "Model S", "color": "white"},
"Porsche": {"year": 2023, "model": "911", "color": "black"}
}
# using pop() to remove a key-value pair from the dictionary
tesla_model = cars.pop("Tesla")
print(cars) # {'BMW': {'year': 2022, 'model': 'M4', 'color': 'red'}, 'Porsche': {'year': 2023, 'model': '911', 'color': 'black'}}
print(tesla_model) # {'year': 2021, 'model': 'Model S', 'color': 'white'}
# using del to remove a key-value pair from the dictionary
del cars["BMW"]
print(cars) # {'Porsche': {'year': 2023, 'model': '911', 'color': 'black'}}
❗With del()
, we cannot store the value of the deleted element as we can with pop()
.
Add New Data
In this section, we will add an additional key-value pair to the "cars" objects. Currently, we only have "topSpeed" represented in miles per hour (mph), but we also want to include the top speed in kilometers per hour (km/h) as "topSpeedKm". We will calculate the "topSpeedKm" by multiplying the "topSpeed" in mph by 1.60934, which will give us the speed in km/h:
# add topSpeed in kmh to the data (1 mph = 1.60934 kmh)
for car in data['cars']:
car['topSpeed'] = {"mph": car['topSpeed'],
"kmh": car['topSpeed'] * 1.60934}
output (partial):
...
{
"manufacturer": "Porsche",
"model": "911 Carrera S",
"year": 2022,
"topSpeed": {
"mph": 192,
"kmh": 308.99328
},
...
As we can see, we were able to add a new key-value pair to our dictionary with some importan piece of information.
Serialization of the data
The term "serialization" refers to the process of converting an object in memory, such as a Python dictionary, into a format that can be stored or transmitted, such as a JSON object. In this context, we can serialize the modified Python dictionary as a JSON object and save it to a file for later use.
After running the following code, a new file called "recoded_cars.json" will be created in the same directory as the JSON file that we have read, containing the serialized JSON object:
# write the data to a file called recoded_cars.json
with open('data/recoded_cars.json', 'w') as f:
f.write(json_string)
Sort Keys alphabetically
With sort_keys()
submethod, we can sort keys.
# write dict to a file
with open('data/sorted.json', 'w') as f:
json.dump(data, f, indent=2, sort_keys=True)
The sort_keys
parameter is used to specify whether the keys in the resulting JSON string should be sorted alphabetically.
If we check the sorted.json file, we will notice that the keys have been sorted:
{
"cars": [
{
"colors": [
{
"code": "#212121",
"name": "Mythos Black Metallic"
},
{
"code": "#a01916",
"name": "Tango Red Metallic"
},
{
"code": "#6c757d",
"name": "Daytona Gray Pearl Effect"
}
],
"drive": "all-wheel",
"engine": {
"horsepower": 602,
"type": "V10"
},
"features": [
"Bang & Olufsen sound system",
"Virtual cockpit",
"Full LED headlights",
"Carbon-ceramic brakes",
"Sport exhaust system"
],
"fuelEconomy": {
"city": 13,
"combined": 16,
"highway": 20
},
...
by setting sort_keys=True
in the json.dump()
method, the keys in the resulting JSON string got sorted alphabetically, resulting in an output where the keys appear in the following order: colors, drive, engine, features, fuelEconomy, and model.
API Requests
Use Case:
X Reseller, a B2B business, recently made a deal with a wholesaler to resell their high-quality widgets. To do so, they requested that the wholesaler provide product data in a standardized format, like JSON, via an API. With this data, X Reseller can create accurate and up-to-date listings on their e-commerce platform and efficiently sell the widgets to other businesses. Using standardized data formats like JSON is an essential part of efficient data exchange in B2B businesses.
❗For this purpose, we will be using DummyJSON that is providing JSON fake data that can be used for generating mock data for prototyping. It can be found on https://dummyjson.com/
❗To make an API request from Python, you need to install the requests
library using the pip
package manager. pip
is a tool for installing and managing Python packages. To install requests
, open a command prompt or terminal window on your computer and type pip install requests
and press enter.
Once the installation is complete, you can use the requests
library in your Python code to send HTTP requests and work with HTTP responses.
As an employee of X Reseller, my next step would be to use the provided API endpoint (https://dummyjson.com/products) to retrieve the product data from the manufacturer. We have been asked to get information about the Iphone phones that the wholesaler is offering.
import requests
import json
# get fake api data from dummyJSON API
# https://dummyapi.io/data/api/user?limit=10
# get the data
response = requests.get("https://dummyjson.com/products")
Now, the response
variable is an object of the Response
class, which is part of the requests
module in Python.
The Response
object contains the server's response to the HTTP request sent by requests.get()
, including the response headers, response body, status code, and other information.
The Response
object has several useful methods and attributes, such as response.status_code
, which returns the HTTP status code of the response, and response.json()
, which parses the response body as JSON and returns a Python object. By accessing these methods and attributes, you can extract the data from the HTTP response and work with it in your Python code.
We will unfortunately not be able to get products just by printing out the response variable. The following examples are request submethods that we can use:
# print different parts of the response
print(response) # <Response [200]>
print(response.status_code) # 200
print(response.headers) # prints out the headers
print(response.text) # prints out the text of the response (json data)
print(response.content) # prints out the content of the response (json data)
print(response.json()) # prints out the json data (same as response.text)
There will be another article about the requests library explaining everything you need to know about it. For now, we need to get the JSON data about the products and response.json()
seems to be a great fit.
We will deserialize the data, as the json()
method of the Response
object is used to convert the response data from its serialized JSON format to a Python object. The json()
method is a convenience method provided by the requests
library that automatically parses the response data as JSON and returns a Python object.
# deserialize the data
data = response.json() # type dict
Now we can print out the data:
{'products': [{'id': 1, 'title': 'iPhone 9', 'description': 'An apple mobile which is nothing like apple', 'price': 549, 'discountPercentage': 12.96, 'rating': 4.69, 'stock': 94, 'brand': 'Apple', 'category': 'smartphones', 'thumbnail': 'https://i.dummyjson.com/data/products/1/thumbnail.jpg', 'images': ['https://i.dummyjson.com/data/products/1/1.jpg', 'https://i.dummyjson.com/data/products/1/2.jpg', 'https://i.dummyjson.com/data/products/1/3.jpg', 'https://i.dummyjson.com/data/products/1/4.jpg', 'https://i.dummyjson.com/data/products/1/thumbnail.jpg']}, {'id': 2, 'title': 'iPhone X', 'description': 'SIM-Free, Model A19211 6.5-inch Super Retina HD display with OLED technology A12 Bionic chip with ...', 'price': 899, 'discountPercentage': 17.94, 'rating': 4.44, 'stock': 34, 'brand': 'Apple', 'category': 'smartphones', 'thumbnail': 'https://i.dummyjson.com/data/products/2/thumbnail.jpg', 'images': ['https://i.dummyjson.com/d
...
Now the data set is pretty large, so it's not possible display it here, but what we are looking for is Iphone phones, so let's try to find all of those. For that, we will be using the Python list comprehension:
# get all the titles of the products via list comprehension
titles = [product['title'] for product in data['products']
if product['title'].startswith('iPhone')]
print(titles)
Output:
['iPhone 9', 'iPhone X']
We can conclude that this wholesaler is providing two Iphone models. Please note that Iphone is spelled with lowecase letter of "i".
We are interested in finding out the price of those two models. We can do that by first finding all iphones and than run an innerloop with list comprehension finding out the price:
# get information about products in titles
for title in titles: # loop through the titles
# print title and price on the same line
print(title, end=': ') # end='' prevents the print function from adding a new line
# print the price
print([product['price'] for product in data['products'] # list comprehension to get the price
if product['title'] == title][0])
If you are unfamiliar with list comprehensions in Python, here comes a short description of the code:
The line starts a for
loop that will iterate over each title
in a list called titles
.
The loop will perform a search for each title
to find matching products.
This line prints the title
of the product to the console, followed by a colon. The end
argument is set to an empty string to prevent the print()
function from adding a new line after the output.
The line print([product['price'] for product in data['products']
uses a list comprehension to find products in the data
JSON object that match the current title
. It creates a list of prices for all products that match the title, and then prints the first price in the list to the console.
Here is the full code:
import requests
import json
# get fake api data from dummyJSON API
# https://dummyapi.io/data/api/user?limit=10
# get the data
response = requests.get("https://dummyjson.com/products")
# deserialise the data
data = response.json() # type dict
#
# get all the titles of the products via list comprehension
titles = [product['title'] for product in data['products']
if product['title'].startswith('iPhone')]
print(titles)
# get information about products in titles
for title in titles: # loop through the titles
# print title and price on the same line
print(title, end=': ') # end='' prevents the print function from adding a new line
# print the price
print([product['price'] for product in data['products'] # list comprehension to get the price
if product['title'] == title][0])
Output:
iPhone 9: 549
iPhone X: 899
Great, we got what we expected.
The code demonstrates a common pattern in working with JSON data, where a loop is used to iterate over a set of data and search for specific values or properties in the JSON object. This can be a powerful technique for processing large volumes of data and extracting relevant information for analysis or other purposes.
Summary
In this article, we have covered the following topics:
- What JSON is
- Data types that we can convert to and from JSON
- Python json module
- Serialization and Deserialization
- Read JSON from a file
- How to perform changes in a JSON
- How to work with API calls using requests library and JSON
About the Author
Aleksandro Matejic, a Cloud Architect, began working in the IT industry over 21 years ago as a technical specialist, right after his studies. Since then, he has worked in various companies and industries in various system engineer and IT architect roles. He currently works on designing Cloud solutions, Kubernetes, and other DevOps technologies.
In his spare time, Aleksandro works on different development projects such as developing devoriales.com, a blog and learning platform launching in 2022/2023. In addition, he likes to read and write technical articles about software development and DevOps methods and tools. You can contact Aleksandro by visiting his LinkedIn Profile.