Query string authentication in Requests
Posted on Fri 11 September 2015 in Code • Tagged with Python, Requests, HTTP, authentication • Leave a comment
Requests is a widely used Python library that provides a nicer API for writing HTTP clients
than the standard urllib2
module does. It deals with authentication in an especially concise way: through
a simple auth=
argument, rather than a
separate password manager & authentication handler or other such nonsense.
There are several possible ways to authenticate an HTTP call with Requests, and it’s pretty easy to implement our own approach if the server requires it. All the built-in ways, however, as well as the examples of custom implementations, are heavily biased towards using HTTP headers to transmit the necessary credentials (such as username/password or some kind of opaque token).
Non-standard auth
This is actually quite reasonable: the most popular authentication methods, including OAuth 1.0 & 2.0, use HTTP headers either primarily or exclusively.
Not every HTTP API follows this convention, though. Sometimes, credentials are put in other parts of the request, commonly the URL itself. It may seem like a bad idea at first but it can also be perfectly acceptable: credentials don’t have to expose secrets of any particular user of the remote system.
Steam API is a good example here. Calling any of its endpoints requires providing an API key but it grants no special rights to access data of any particular Steam user. All the information it returns is already visible on their public profile1.
For those special authentication schemes, Requests necessitate rolling out our own implementation. Thankfully, doing so is mostly pretty straightforward.
Simple example
All Requests’ authenticators are callable objects inheriting from requests.auth.AuthBase
class.
Writing your own is hence a matter of defining a subclass of AuthBase
with at least a __call__
method:
class SillyAuth(AuthBase):
def __call__(self, request):
request.headers['X-ID'] = 'im valid so auth is yes'
return request
# usage
requests.get('http://example.com', auth=SillyAuth())
The job of an authenticator is to modify the request
so that it includes appropriate credentials in whatever form
necessary to have them accepted by the remote server. Like I’ve mentioned before, HTTP headers are the most common option,
but the request can be modified in other ways as well.
Query string parameters
One problem with modifying a query string, though, is that it’s a part of request URL. By the time it reaches authenticators, the Requests library has already merged any additional query params into it2. Including more params will thus require modifying the URL.
Though it may sound like a risky endeavour involving string manipulations that are fraught with security issues, it’s not really that bad at all. In fact, the Requests library provides an API to do exactly this:
class QueryStringAuth(AuthBase):
"""Authenticator that attaches a set of query string parameters
(e.g. an API key) to the request.
"""
def __init__(self, **params):
self.params = params
def __call__(self, request):
if self.params:
request.prepare_url(request.url, self.params)
return request
Albeit scantly documented,
the prepare_url
method will take an existing URL and a dictionary of query string params, and outfit the request
with a brand new URL that contains those params neatly encoded.
Full implementation of QueryStringAuth
is a little more involved
than the snippet above, because we should like to replicate all the idiosyncracies of how
regular Requests API
handles query string params. Some of them — like allowing both strings and lists as param values — are taken care of
by prepare_url
itself, but the rest should be dealt with on our own.
Usage
To finish up, let’s use this authenticator to call Steam API and return a list of games that a given user owns but hasn’t played yet:
STEAM_API_KEY = 'a1b2c3d4e5f6g7h8i9j' # not a real one, of course
def get_steam_backlog(steam_id):
url = 'http://api.steampowered.com/IPlayerService/GetOwnedGames/v0001/'
params = {
'steamid': steam_id,
'include_appinfo': 1,
}
response = requests.get(
url, params=params, auth=QueryStringAuth(key=STEAM_API_KEY))
games = response.json().get('response', {}).get('games', ())
for game in games:
if game.get('playtime_forever', 0):
continue
yield game['name']
We could’ve put STEAM_API_KEY
directly in params
, of course. Singling it out explicitly as an authentication
detail, however, makes the code clearer and plays nicely with more advanced features of Requests,
such as sessions.
-
It can be said that only in this case we’re dealing with exclusively authentication, whereas the others also perform authorization. I wouldn’t quibble too much about those details. The fact that both terms are often shortened to “auth” doesn’t exactly help with distinguishing them anyway. ↩
-
In fact, what
AuthBase.__call__
receives is a specialPreparedRequest
object which contains the exact bytes that’ll be sent to the server. Most of the higher level abstractions offered by the Requests library (like form data or JSON request body) has been compiled to raw octets at this point. This is done to allow some authenticators (like OAuth) to analyze the full request payload and sign it cryptographically as a part of their flow. ↩