How to download data in multiple file formats (CSV, XLS, TXT) with Django REST Framework
A lot of the time, our typical server response is in the form of JSON or XML. This serves our use cases a good number of times, however, there are times when the need to provide data in the form of a file arises.
In this article, we will be exploring how to convert our model data into files and send them as responses in Django REST Framework (DRF). We will do this by building out a simple project.
Prerequisites
You should have some basic Django and Django REST Framework knowledge. You should also have python already installed on your system.
The project, Setting up and Installation
We will be building out a simple Student Management App. The purpose of the app will be to:
Allow a user to input student data
Download all the available data as either a CSV, Excel or TXT file.
Creating the project and Installing Dependencies
We will install Django and DRF to start. To do this, first, create your virtual environment and activate it.
python3 -m venv studentappenv
source studentappenv/bin/activate
Now install Django and DRF. We will also be installing openpyxl
which is a python package that helps when interacting with data and excel files.
pip install django djangorestframework openpyxl
Next, create the Django project and create an app.
django_admin startproject student_management .
python manage.py startapp student_data
Update your settings.py and include the new app inside the INSTALLED_APPS
list.
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'student_data', #new line
]
Let's create our model. Head over to student_data/models.py
from django.db import models
class StudentData(models.Model):
STUDENT_GENDER = ((1, 'Male'), (2, 'Female'), (3, 'Other'))
STUDENT_LEVEL = ((1, 'Junior'), (2, 'Senior'))
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
age = models.IntegerField()
gender = models.IntegerField(choices=STUDENT_GENDER)
level = models.IntegerField(choices=STUDENT_LEVEL)
A pretty simple model that stores basic student information. This will be good enough for our needs.
Next up, we create our serializer. Create a new file serializers.py
inside the student_data
folder and add the following inside.
from rest_framework import serializers
from .models import StudentData
class StudentDataSerializer(serializers.ModelSerializer):
class Meta:
model = StudentData
fields = '__all__'
def to_representation(self, instance):
data = super().to_representation(instance)
data['gender'] = instance.get_gender_display()
data['level'] = instance.get_level_display()
return data
This is also a basic serializer for our StudentData
model. We override our to_representation
to change how the server would return values for the gender
and level
fields. Instead of returning something like 'gender': '1'
, we will instead get 'gender': 'Male'
.
We have our model and serializer, now, let's create our view. Inside student_data/views.py
, add the following lines of code:
from django.utils import timezone
from rest_framework.response import Response
from rest_framework.viewsets import ModelViewSet
from rest_framework.decorators import action
from .models import StudentData
from .serializers import StudentDataSerializer
class StudentDataViewset(ModelViewSet):
serializer_class = StudentDataSerializer
queryset = StudentData.objects.all()
@action(detail=False, methods=["get"])
def download(self, request):
queryset = self.get_queryset()
serializer = StudentDataSerializer(queryset, many=True)
return Response(serializer.data)
This is our view which will handle POST
requests to create our student data. We have also defined a new viewset action download
which we will use for downloading our file responses. Right now it will simply return the data as a JSON response, we will tweak it soon to meet our needs.
Let's connect our URL paths. Create a urls.py
file in the student_data
folder.
# student_data/urls.py
from .views import StudentDataViewset
from django.urls import path, include
from rest_framework import routers
router = routers.DefaultRouter()
router.register("", StudentDataViewset, basename="student-data")
urlpatterns = [
path("", include(router.urls)),
]
Connect this to the main urls file at student_management
folder.
# student_management/urls.py
from django.contrib import admin
from django.urls import path, include
urlpatterns = [
path('admin/', admin.site.urls),
path('student-data/', include('student_data.urls')),
]
Run makemigrations and migrate
python manage.py makemigrations
python manage.py migrate
Start the server with python manage.py runserver
.
Make a few POST
requests at http://127.0.0.1:8000/student-data/
to populate our database with some data.
Now test the http://127.0.0.1:8000/student-data/download/
endpoint with a GET
request to see the response type. You will get JSON data as a response similar to what we saw from our POST
request. This is because DRF sets a global default renderer rest_framework.renderers.JSONRenderer
which will return server responses in application/json
media type.
We will create our own renderers which will convert our data into the response formats we want.
Create a renderers.py
file inside the student_data
folder.
# student_data/renderers.py
import io
import csv
from rest_framework import renderers
STUDENT_DATA_FILE_HEADERS = ["id", "first_name", "last_name", "age", "gender", "level"]
class CSVStudentDataRenderer(renderers.BaseRenderer):
media_type = "text/csv"
format = "csv"
def render(self, data, accepted_media_type=None, renderer_context=None):
csv_buffer = io.StringIO()
csv_writer = csv.DictWriter(csv_buffer, fieldnames=STUDENT_DATA_FILE_HEADERS, extrasaction="ignore")
csv_writer.writeheader()
for student_data in data:
csv_writer.writerow(student_data)
return csv_buffer.getvalue()
This is going to be the renderer that will convert our data into CSV format. Let's break this code down.
We set a
STUDENT_DATA_FILE_HEADERS
list which contains all our model fields.Our
CSVStudentDataRenderer
class subclassesBaseRenderer
and defines themedia_type
andformat
attributes. We have many different media types which you can find here at IANA defined media types (mime types).We define a render method where a bulk of the work is done. Inside it:
We create a string buffer named
csv_buffer
.We create an instance of
csv.DictWriter
and pass in our buffer as an argument.We call
csv_writer.writeheader()
which will write the values inSTUDENT_DATA_FILE_HEADERS
as the first line of our CSV file.We use the
csv
helper module to write our data as comma separated values in the buffer location.We retrieve the content written into our buffer with
csv_buffer.getvalue()
Let's add renderers for Excel and Text files next. Update renderers.py
file.
import io
import csv
import openpyxl # new
from rest_framework import renderers
STUDENT_DATA_FILE_HEADERS = ["id", "first_name", "last_name", "age", "gender", "level"]
...
# ------ NEW LINES --------
class TextStudentDataRenderer(renderers.BaseRenderer):
media_type = "text/plain"
format = "txt"
def render(self, data, accepted_media_type=None, renderer_context=None):
text_buffer = io.StringIO()
text_buffer.write(' '.join(header for header in STUDENT_DATA_FILE_HEADERS) + '\n')
for student_data in data:
text_buffer.write(' '.join(str(sd) for sd in list(student_data.values())) + '\n')
return text_buffer.getvalue()
class ExcelStudentDataRenderer(renderers.BaseRenderer):
media_type = "application/vnd.ms-excel"
format = "xls"
def render(self, data, accepted_media_type=None, renderer_context=None):
workbook = openpyxl.Workbook()
buffer = io.BytesIO()
worksheet = workbook.active
worksheet.append(STUDENT_DATA_FILE_HEADERS)
for student_data in data:
worksheet.append(list(student_data.values()))
workbook.save(buffer)
return buffer.getvalue()
The idea is the same as our CSVStudentDataRenderer
. We define the right media_type
and format
of our expected file. We then create a buffer which we write our headers and model data into. After that, we return the content of the buffer. In the case of our ExcelRenderer
, we use the openpyxl
library to simplify the buffer writing process.
Now that we have our renderers, let's update our view to use it. Update your student_data/views.py
file.
from django.utils import timezone
from rest_framework.response import Response
from rest_framework.viewsets import ModelViewSet
from rest_framework.decorators import action
from .models import StudentData
from .renderers import CSVStudentDataRenderer, ExcelStudentDataRenderer, TextStudentDataRenderer
from .serializers import StudentDataSerializer
class StudentDataViewset(ModelViewSet):
serializer_class = StudentDataSerializer
queryset = StudentData.objects.all()
@action(detail=False, methods=["get"], renderer_classes=[CSVStudentDataRenderer, ExcelStudentDataRenderer, TextStudentDataRenderer])
def download(self, request):
queryset = self.get_queryset()
now = timezone.now()
file_name = f"student_data_archive_{now:%Y-%m-%d_%H-%M-%S}.{request.accepted_renderer.format}"
serializer = StudentDataSerializer(queryset, many=True)
return Response(serializer.data, headers={"Content-Disposition": f'attachment; filename="{file_name}"'})
The three main takeaways from the new changes are:
We define a renderers_classes parameter in our
@action
decorator. This will be used by thedownload
endpoint when choosing a renderer. How will it decide? we will talk about that in a minute.We create a distinct file name based on the current time and append the format of the renderer which would be used to serve the particular response, found at
request.accepted_renderer.format
.We set a
content-dispostion
header which lets the client know that the response should be treated as an attachment that should be downloaded under the value offilename
.
Let's head back to postman to test our download
endpoint.
I get a CSV file downloaded onto my system.
Great. This means Django decided to use our CSVRenderer
, but how did it decide? Django has a content negotiation mechanism which it uses to determine how to render a response back to the client. It looks into the Accept
header in the request and tries to map it to an available renderer. If it is unable to map it to any specific one, by default, Django chooses the first renderer in the renderers_classes
list. This is why we get a CSV file in return.
Let's edit our Accept
request header and specify the Excel media type(application/vnd.ms-excel
) then retry the request.
I now get an Excel file downloaded.
And if we update the Accept
header to text/plain
we get a txt file downloaded as Django chooses the text renderer class.
That's it.
Conclusion
In this article, we learned how to create our own custom renderers for specific media types and formats like excel, CSV and text. We also saw how to help our application pick the right renderer using the Accept
request header.
If you found this article useful or learned something new, consider dropping a heart up and following me to keep up-to-date with any recent postings!
You can also find me on Twitter at akinsola232 and on LinkedIn at Ademola.
Till next time, happy coding!
Levi