File handling in Python is an essential aspect of programming. This allows us to store (create), retrieve (read), and manipulate (write) data in files. Python provides built-in functions and methods to work with files efficiently. In this tutorial, we’ll cover the basics of file handling in Python, including opening, reading, writing, and closing of files. We will also cover advanced concepts like working with different file modes, handling exceptions, and using file context managers.
File handling in Python
In Python, file handling refers to the process of performing operations like creating, reading, writing, and closing of files. Python provide built-in functions to perform these operations on files. Python supports two file types:
- Text files (.txt, .csv, etc.)
- Binary files (images, videos, executable, etc.)
File Modes in Python
While opening a file in Python, we need to specify the mode that determines how the file will be handled and what operations we can perform on that file. Python provides multiple file modes:
'r'– Read mode (default). This opens the file for reading. This mode will raiseFileNotFoundError, If the file doesn’t exist.'w'– Write mode. This opens the file for writing, overwriting if the file exists. This mode creates a new file, if it doesn’t exist. It truncates the file first.'a'– Append mode. This opens the file for appending data at the seek position. On some Unix systems, means that all writes append to the end of the file regardless of the current seek position'x'– Exclusive creation mode. This creates a new file and raisesFileExistsErrorerror if it already exists.'t'– Text mode (default). This opens the file in text mode. In text mode, if encoding is not specified the encoding used is platform dependent:locale.getencoding()is called to get the current locale encoding.'rt'becomes the combined default mode.'b'– Binary mode. Used with other modes to read or write binary files (e.g.,'rb','wb').'rb'– Read and Binary mode. This is used to read binary files.'wb'– Write and Binary mode. This is used to write binary files.'+'– This opens a disk file for updating (reading and writing)
Python distinguishes between files opened in binary and text modes, even when the underlying operating system doesn’t. Files opened in binary mode return contents as bytes objects without any decoding. In text mode the contents of the file are returned as strings, the bytes having been first decoded using a platform-dependent encoding or using the specified encoding if given.
Python’s built-in open() function is used to open a file, while other methods like read(), write(), and close() are used to manipulate the file.
Open files in Text and Binary mode
In Python, files are opened using the open() built-in function. We can open the file in different modes using open() function. Following is the syntax for open() function.
open(name, mode=None, buffering=None)
open()function opens the file and return a stream. Raise OSError upon failure.name– This is either the name of the file present in current working directory or the path of the file, if file isn’t in the current working directory.mode– This is an optional string that specifies the mode in which the file is opened. It defaults to'rt'which means open for reading in text mode.buffering– This is an optional integer used to set the buffering policy.0is used to switch buffering off (only allowed in binary mode),1to select line buffering (only usable in text mode), and aninteger > 1to indicate the size of a fixed-size chunk buffer.
Open files in Text mode
# Scenario 1 - open a file with read and text mode
print("open a file with read and text mode. File should exist")
file_read_text = open('course_description.txt', 'r')
# Scenario 2 - read mode - error when file does not exists
print("read mode - error when file does not exists")
try:
file_read_text = open('course_details.txt', 'r')
except FileNotFoundError as err:
print("error", err)
# Scenario 3 - open file in write and text mode
# if it does not exist, create an empty file with the given name
print("open file in write and text mode")
print("if it does not exist, create an empty file with the given name")
file_write_text = open('course_details.txt', 'w')
# Scenario 4 - append and text mode, creates the file if it does not exist
print("append and text mode, creates the file if it does not exist")
file_append_text = open('course_details.txt', 'a')
# Scenario 5 - create an empty file with the given name, returns an error if the file exists
print("create an empty file with the given name, returns an error if the file exists")
try:
file_create_text = open('course_details.txt', 'x')
except FileExistsError as err:
print("error", err)open('course_description.txt', 'r')– This opens the filecourse_description.txtin read and text mode. This file should be present in current working directory.'rt'is the default mode.- In second scenario,
open('course_details.txt', 'r')will raiseFileNotFoundErrorif the filecourse_details.txtdoes not present in current working directory. open('course_details.txt', 'w')– This opens the filecourse_details.txtin write and text mode. It will create the file, if file does not exists in the current working directory.'t'is the default mode.open('course_details.txt', 'a')– This opens the filecourse_details.txtin append and text mode. It will create the file, if file does not exists in the current working directory.'t'is the default mode.open('course_details.txt', 'x')– This creates an empty filecourse_details.txtin the current working directory. This will raiseFileExistsErrorif file already exists.
Open files in Binary mode
# open a file with read and binary mode
print("open a file with read and binary mode. File should exist")
file_read_binary = open('course_description.txt', 'rb')
# open a file from differnt path, with read and binary mode
file_read_binary = open('D://shbytes/course_description.txt', 'rb')
# open a file in write and binary mode
# if it does not exist, create an empty file with the given name
print("open a file in write and binary mode")
print("if it does not exist, create an empty file with the given name")
file_write_binary = open('course_description.txt', 'wb')
# append and binary mode, creates the file if it does not exist
print("append and binary mode, creates the file if it does not exist")
file_append_binary = open('course_details.txt', 'ab')open('course_description.txt', 'rb')– This opens the filecourse_description.txtin read and binary mode. This file should be present in current working directory.open('D://shbytes/course_description.txt', 'rb')– This opens the file from a different path. We can use the file path, if file not present in current working directory.open('course_description.txt', 'wb')– This opens the filecourse_description.txtin write and binary mode. It will create the file, if file does not exists in the current working directory.open('course_details.txt', 'ab')– This opens the filecourse_details.txtin append and binary mode. It will create the file, if file does not exists in the current working directory.
Close files
In Python, files are closed using the close() method. When we open the file using the open() function, it uses system resources and files are locked for a particular operation. The file and system resources are not free until we close the opened file. It’s important to close a file after we are done to free up system resources.
close() method is called on opened file object like open_file_object.close().
# close all the open files
# always close the files; because of security & buffering issues, changes may not show until close of the file
print("close all the open files")
file_read_text.close() # close file_read_text object
file_read_binary.close() # close file_read_binary object
file_write_text.close() # close file_write_text object
file_append_text.close() # close file_append_text object
try:
file_create_text.close() # if object does not exist, give NameError
except NameError as err:
print("error", err)- With
open()function program, we opened multiple files and created open file objects. We can callclose()method on those file object. file_create_text.close()givenNameError, if the object does not exist.- We cannot perform operations on file after closing it. We have open the file again to perform operations on that file.
Read from files in Text and Binary mode
After opening the file using open() function, we can read the file content. Python provides several ways to read from files.
read(size)– Reads the specified number of characters from the seek position.readline()– Reads a single line from the file from the seek position.readlines()– Reads all lines in the file and returns them as a list.
Let’s assume we have created a file course_description.txt with the sample content written line by line:
Python is a powerful language for data manipulation and file handling.It pairs seamlessly with Power BI, Microsoft’s leading data visualization tool.In this tutorial, we will learn how to read data from files using PythonClean or process it, and then connect the resulting data to Power BI for visualization.Useful when dealing with large datasets, log files that need processing before analysis.
Read files in Text mode
We will read the content from this file using the different read functions:
# open a file with read and text mode
print("open a file with read and text mode")
file_read_text = open('course_description.txt', 'r')
# Read first 9 characters from the file text
print(file_read_text.read(9)) # Output => Python is
# Read next 20 characters starting from seek position
print(file_read_text.read(20)) # Output => a powerful language
# Read next line of the file starting from seek position
print(file_read_text.readline()) # Output => for data manipulation and file handling
# Read next line of the file starting from seek position
print(file_read_text.readline()) # Output => It pairs seamlessly with Power BI, Microsoft's leading data visualization tool.
# Read complete content of the file starting from seek position
print(file_read_text.read())
# Output
# In this tutorial, we will learn how to read data from files using Python
# Clean or process it, and then connect the resulting data to Power BI for visualization.
# Useful when dealing with large datasets, log files that need processing before analysis.
for line in file_read_text: # Read file content line by line in loop
print(line)
file_read_text.close()Follow the program steps:
- Using
open('course_description.txt', 'r')– we opened the file in read and text mode, referenced with variablefile_read_text. Seek (cursor) position will be at the start. file_read_text.read(9)– This will read next 9 characters from the file text starting from current seek position. In this case seek position is at start, so it will read first 9 characters.file_read_text.read(20)– This will read next 20 characters from the file text starting from current seek position. From last case, seek position is at character 9 so it will read next 20 character.file_read_text.readline()– This will read line of the file starting from seek position. From last case seek position will at character 29 (20+9) and will read the first line from there. We usedreadline()two times to read two lines.file_read_text.read()– This will read the entire content of the file, starting from seek position. From last case, seek position will be at third line. So it will read all content from third line.- Using
forloop, we can read all content of the file line by line. - At end, close the file read object
file_read_text.close()
Read files in Binary mode
Similarly, we can open the file with read and binary mode. We can use read(size), readline() or readlines() method to read the content in binary mode as well.
# open a file with read and binary mode
print("open a file with read and binary mode")
file_read_binary = open('course_description.txt', 'rb')
# Read first 9 characters from the file
print(file_read_binary.read(9)) # Output => b'Python is'
# Read next 20 characters starting from seek position
print(file_read_binary.read(20)) # Output => b' a powerful language'
# Read next line of the file starting from seek position
print(file_read_binary.readline()) # Output => b' for data manipulation and file handling\r\n'
# Read next line of the file starting from seek position
print(file_read_binary.readline())
print(file_read_binary.read())
for line in file_read_binary: # Read file content line by line in loop
print(line)Write and Append to Files
Create an empty file
# create an empty file with the given name, returns an error if the file exists
print("create an empty file with the given name, returns an error if the file exists")
try:
file_create_text = open('course_details.txt', 'x')
except FileExistsError as err:
print("error", err)Write to a file
To write data to a file, open the file in 'w' mode. This will overwrite any existing content.
# open file in write and text mode
# if it does not exist, create an empty file with the given name
print("open file in write and text mode")
print("if it does not exist, create an empty file with the given name")
file_write_text = open('course_details.txt', 'w')
file_write_text.write("Writing text to the file in write mode") # writing first line to the file
file_write_text.write("\nPython Learning @shbytes!!") # writing second line to the file
file_write_text.close() # close the file write object
file_read_text = open('course_details.txt', "r")
print(file_read_text.read()) # read the entire content of the file
file_read_text.close() # close the file read object open('course_details.txt', 'w')– This will open the file in write and text mode. It will create the file, if it doesn’t exists.file_write_text.write("\nPython Learning @shbytes!!")– This writes a new line into the file.\nis used to create a new line in the content.file_write_text.close()– close the file write object- Similarly, we can use
open('course_details.txt', 'wb')to open the file in write and binary mode. Then we can write into the file using thewrite()method.
Append to a file
To add content at the end of the file without overwriting, use the 'a' mode.
# open file in append and text mode
# if it does not exist, create an empty file with the given name
print("open file in append and text mode")
print("if it does not exist, create an empty file with the given name")
file_append_text = open('course_details.txt', 'a') # open the file in append mode
file_append_text.write("Writing text to the file in append mode") # add a line to the file
file_append_text.write("\nPython is easy to learn!!") # add another line to the file
file_append_text.close() # close the file append object
file_read_text = open('course_details.txt', "r") # open the file read mode
print(file_read_text.read()) # read entire content of the file
file_read_text.close() # close the file read objectopen('course_details.txt', 'a')– This will open the file in append mode. It will create a new file, if it doesn’t exists.file_append_text.write("\nPython is easy to learn!!")– This writes a new line into the file.\nis used to create a new line in the content.file_append_text.close()– close the file append object- Similarly, we can use
open('course_details.txt', 'ab')to open the file in append and binary mode. Then we can write into the file using thewrite()method.
Using with statement
The with statement simplifies file handling by automatically closing the file after the block of code is executed. This reduces the risk of leaving files open unintentionally.
with statement to read file
# Using 'with' to handle files
# with - to open and read file in one command
print("with - to open and read file in one command")
with open("course_description.txt") as file_read_text: # open file in read mode
file_data = file_read_text.read() # read entire content of the file
print(file_data)
# No need to manually close the file; it's done automaticallywith open("course_description.txt") as file_read_text: => This statement open the file in read mode and assign it as file read object. Then we can use that file object to read the file content. We don’t need to manually close the file object, it will be closed automatically.
with statement to write file
# with - to open and write file in one command
print("with - to open and write file in one command")
with open('course_details.txt', 'w') as file_write_text: # open file in write mode
file_write_text.write("\nWriting text to the file in write mode") # write content to the file
file_write_text.write("\nPython Learning @shbytes!!")
# No need to manually close the file; it's done automaticallywith open('course_details.txt', 'w') as file_write_text: => This statement open the file in write mode and assign it as file write object. Then we can use that file object to write content to the file. We don’t need to manually close the file object, it will be closed automatically.
File Other Methods
There are multiple other methods that we can use with the file read object and file write object.
| Method Syntax | Description |
|---|---|
file.fileno() | Method to return the file descriptor of the stream, as a number |
file.isatty() | Method returns True if file stream is interactive |
file.readable() | Method returns True if the file is readable, else False |
file.seekable() | Method returns True if file is seekable, else False |
file.seek(offset) | Method sets current file position in a file stream and returns the new position |
file.tell() | Method returns the current file position in file stream |
file.writable() | Method returns True if the file is writable, else False |
file.closed | True if file is closed, else False |
file.mode | Access mode with which file was opened |
file.name | Name of the file |
file.flush() | Method to clean the internal buffer |
file.truncate(size) | Method resizes the file to given number of bytes |
file.writelines(list) | Method writes list of items to the file |
file_read_text = open("course_details.txt", "r")
# file.fileno() - method to return the file descriptor of the stream, as a number
print("fileno of the file - ", file_read_text.fileno()) # Output => 3
# file.isatty() - method returns True if file stream is interactive
print("file stream is interactive - ", file_read_text.isatty()) # Output => False
# file.readable() - method returns True if the file is readable, else False
print("file is readable - ", file_read_text.readable()) # Output => True
# file.seekable() - method returns True if file is seekable, else False
print("file is seekable - ", file_read_text.seekable()) # Output => True
# file.seek(offset) - method sets current file position in a file stream and returns the new position
print("file seek offset - ", file_read_text.seek(4)) # # Output => 4
# file.tell() - method returns the current file position in file stream
print("file tell current position - ", file_read_text.tell()) # Output => 4
# file.writable() - method returns True if the file is writable, else False
print("file is writable - ", file_read_text.writable()) # Output => False
# file.closed - True if file is closed, else False
print(file_read_text.closed) # Output => False
# file.mode - access mode with which file was opened
print(file_read_text.mode) # Output => r
# file.name - name of the file
print(file_read_text.name) # Output => course_details.txt
file_read_text.close() # close file objectfile_write_text = open('course_details.txt', 'w')
# file.flush() - method to clean the internal buffer
file_write_text.write("some random text")
file_write_text.flush()
# file.truncate(size) - method resizes the file to given number of bytes
print(file_write_text.truncate(5)) # Output => 5
# file.writable() - method returns True if the file is writable, else False
print("file is writable - ", file_write_text.writable()) # Output => True
# file.writelines(list) - method writes list of items to the file
file_write_text.writelines(["Python Learning @shbytes!!","\nNew line"])
file_write_text.close() # close file objectHandling File Exceptions
It’s good practice to handle errors that may occur during file operations, such as attempting to read a non-existent file. This can be done using try-except blocks.
try:
with open('non_existent.txt', 'r') as file:
content = file.read()
except FileNotFoundError:
print("File not found. Please check the file name.")Summary
In this article, we learned about File handling in Python. Following topics were discussed:
- File handling in Python
- Open files in Text and Binary mode
- Close files
- Read from files in Text and Binary mode
- Write and Append to Files
- Using with statement
- File Other Methods
- Handling File Exceptions
Code – Github Repository
All code snippets and programs for this article and for Python tutorial, can be accessed from Github repository – Comments and Docstring in Python.