Python Read and Write to a Properties File : Chris
by: Chris
blow post content copied from Be on the Right Side of Change
click here to view original post
Properties files are simple text files used for configuring parameters and settings. They store data in key-value pairs, so you need them to set up environment-specific configurations when creating software projects.
Example Property File
Here’s an example of a properties file for a web scraping application. This file contains key-value pairs defining various configurations required for the web scraping process.
# Web Scraping Application Properties # URL to scrape target_url = https://example.com # Time in seconds to wait between requests to avoid server overload request_interval = 2 # User agent to simulate browser behavior user_agent = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3 # Maximum number of retries for a failed request max_retries = 3 # Timeout in seconds for web requests request_timeout = 5 # Proxy settings (if required) proxy_enabled = False proxy_url = http://proxyserver:port # Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL) log_level = INFO
In this properties file:
target_url
specifies the URL to be scraped.request_interval
ensures a delay between requests to prevent overloading the server.user_agent
defines the user agent string to be used for requests, mimicking a real browser.max_retries
sets the number of attempts for a failed request.request_timeout
is the time limit for each request.proxy_enabled
andproxy_url
are used if accessing the target URL requires a proxy.log_level
sets the verbosity of log messages.
These settings are adjustable based on the specific needs of the web scraping application.
Reading a Properties File in Python
Python’s built-in functions make reading from a properties file straightforward.
def read_properties(file_path): properties = {} with open(file_path, 'r') as file: for line in file: if line.startswith('#') or not line.strip(): continue # Skip comments and blank lines key, value = line.strip().split('=', 1) properties[key] = value return properties
This function reads each line, skips comments and empty lines, and extracts key-value pairs.
When printed directly, the output will be in Python’s dictionary format. Here’s how the printed output of the properties
dictionary would look:
{ 'target_url': 'https://example.com', 'request_interval': '2', 'user_agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3', 'max_retries': '3', 'request_timeout': '5', 'proxy_enabled': 'False', 'proxy_url': 'http://proxyserver:port', 'log_level': 'INFO' }
This format clearly shows each key-value pair as a Python dictionary, which is very useful for further processing or configuration within a Python script.
Modifying a Properties File
Altering a properties file involves reading the existing content, modifying it, and writing it back.
def modify_property(file_path, key, new_value): properties = read_properties(file_path) properties[key] = new_value with open(file_path, 'w') as file: for key, value in properties.items(): file.write(f'{key}={value}\n')
Here, we read the properties, update the relevant key, and write back the modified content.
Putting It All Together
Here’s the complete Python script that combines everything: creating a mock properties file, reading the properties with the read_properties
function, and then printing out the properties in a readable format.
This serves as a quick and easy example of how to handle properties files in Python for a web scraping application.
# Python Script to Read Properties from a File def read_properties(file_path): """ Reads a properties file and returns the properties as a dictionary. """ properties = {} with open(file_path, 'r') as file: for line in file: line = line.strip() if line.startswith('#') or not line: continue # Skip comments and blank lines key, value = [part.strip() for part in line.split('=', 1)] properties[key] = value return properties # Example Properties File Content properties_content = """ # Web Scraping Application Properties # URL to scrape target_url = https://example.com # Time in seconds to wait between requests to avoid server overload request_interval = 2 # User agent to simulate browser behavior user_agent = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3 # Maximum number of retries for a failed request max_retries = 3 # Timeout in seconds for web requests request_timeout = 5 # Proxy settings (if required) proxy_enabled = False proxy_url = http://proxyserver:port # Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL) log_level = INFO """ # Write the properties content to a temporary file import tempfile with tempfile.NamedTemporaryFile(delete=False, mode='w') as temp_file: temp_file.write(properties_content) temp_file_path = temp_file.name # Read the properties from the file properties = read_properties(temp_file_path) # Print the properties print("Properties read from the file:") for key, value in properties.items(): print(f"{key}: {value}") # Optionally, you can delete the temporary file after reading import os os.remove(temp_file_path)
This script:
- Defines the
read_properties
function for reading properties files. - Creates a mock properties file with relevant content for a web scraping application.
- Writes this content to a temporary file.
- Reads the properties from the file using
read_properties
. - Prints the properties in a readable format.
- Deletes the temporary file after use.
You can run this script as is to see how it processes the properties file. The output will display the parsed key-value pairs from the properties content.
December 18, 2023 at 04:10AM
Click here for more details...
=============================
The original post is available in Be on the Right Side of Change by Chris
this post has been published as it is through automation. Automation script brings all the top bloggers post under a single umbrella.
The purpose of this blog, Follow the top Salesforce bloggers and collect all blogs in a single place through automation.
============================
Post a Comment