class MyUser(User):
    job = ...
    city = ....

MyUser.objects.get(pk=request.user) and MyUser.objects.get(pk=request.user.id) give me the same result. From doc, pk must be an int. But request.user is an object. Why queries are the same for object and user id?

edit : I know that request.user is object but I want to know why results are same.

Thanks in advance

This is by design - if you pass a model object as the parameter to a query, it is the same as passing its primary key.

If this wasn't the behaviour, either passing the pk would be required, or an error, which would merely be annoying.

Thanks to rebus for this reference to the source: https://code.djangoproject.com/browser/django/trunk/django/db/models/fields/related.py#L175

I am in the process of learning Python and had a question about the future. I know it's not the most pressing thing to think about currently, but I'm curious.

Currently, Django only supports up to Python 2.7. However, in the near future, it will be supporting Python 3. In terms of writing code in Python 2.7 and using the related Django framework, what happens when the transition to Python 3 actually comes along.

Presumably, I'd learn and code in the newer version. However, what about maintaining the old code? Does it stay as is? Does it need to be rewritten?

I'm just curious as to how these transition work. Also, does it make a difference that Python 3 isn't backwards-compatible? What is the consequence of that? For instance, I read that Ruby versions 1.8 to 1.9 (and even the future 2.x) were backwards-compatible and less of a leap (than Python 2.x to 3.x). I wonder if that split between Python versions creates any fragmentation problems or code maintenance problems?

So, if someone could try explaining to me what goes on with these updates and the issues at hand when dealing with them, I'd really appreciate it. Thanks!

... what happens when the transition to Python 3 actually comes along.

Nothing. That's why you're using a framework in the first place. All you need to be responsible for is the small amount of your code that will need to be ported.

An Android/Iphone app will be accessing application data from the server. [Django-Python]

How can I secure the communication with the mobile app ?

Expectation : Secure enough for sensitive information like passwords, there shall be no direct way of decryption except brute-forcing.

My requirements :

• Authentication [Only the app is authorized]
• Integrity [Messages should not be modified in between]
• Privacy [Communication should not be readable if sniffed]

My effort:

• SSL authenticates only the Server, not the client.
• I can-not use a symmetric encryption [Provides only Privacy]
• Digital signature is not possible [Lacks Privacy]
• PGP full-fills all 3 requirements.

Problem :

• PGP requires to store keys on client app.
• There seems to be no assuring way of securing keys on client app.
• If the key is out, then PGP or Symmetric encryption are equally vulnerable.
• Reverse-Engineering PGP keys or symmetic keys is equally hard.
• In that case PGP is a non-sense burden on the mobile processor.
• OAuth is again useless, since it also have a client key.

So, how can/should I move forward on this ? How does the industry deals with this ?

Should I implement casual approach :

• Use simple SSL and cross my fingers ?, since authentication is not possible if the keys are stolen? (Only server authentication is possible with this)

You're working on bad information. SSL can absolutely authenticate the client, it's just not something that is done for the bulk of SSL as the protocol is (or, atleast was) typically used to protect e-commerce sites where authentication of the server was important but doing so with the client was not important and/or not feasible. What you want to do is employ mutually-authenticated SSL, so that your server will only accept incoming connections from your app and your app will only communicate with your server.

Here's the high-level approach. Create a self-signed server SSL certificate and deploy on your web server. If you're using Android, you can use the keytool included with the Android SDK for this purpose; if you're using another app platform like iOS, similar tools exist for them as well. Then create a self-signed client and deploy that within your application in a custom keystore included in your application as a resource (keytool will generate this as well). Configure the server to require client-side SSL authentication and to only accept the client certificate you generated. Configure the client to use that client-side certificate to identify itself and only accept the one server-side certificate you installed on your server for that part of it.

If someone/something other than your app attempts to connect to your server, the SSL connection will not be created, as the server will reject incoming SSL connections that do not present the client certificate that you have included in your app.

A step-by-step for this is a much longer answer than is warranted here. I would suggest doing this in stages as there are resources on the web about how to deal with self-signed SSL certificate in both Android and iOS, both server and client side. There is also a complete walk-through in my book, Application Security for the Android Platform, published by O'Reilly.

I have a complex database model set up in Django, and I have to do a number of calculations based on filter data. I have a Test object, a TestAttempt object, and a UserProfile object (with a foreign key back to test and a foreign key back to a userprofile). There is a method that I run on a TestAttempt that calculates the test score (based on a number of user-supplied choices compared to the correct answers associated with each test). And then another method that I run on a Test that calculates the average test score based on each of its associated TestAttempt's But sometimes I only want the average based on a supplied subset of the associated TestAttempt's that are linked with a particular set of UserProfiles. So instead of calculating the average test score for a particular test this way:

[x.score() for x in self.test_attempts.all()]

and then averaging these values. I do a query like this:

[x.score() for x in self.test_attempts.filter(profile__id__in=user_id_list).all()]

where user_id_list is a particular subset of UserProfile id's for which I want to find the average test score in the form of a list. My question is this: if user_id_list is indeed the entire set of UserProfile's (so the filter will return the same as self.test_attempts.all()) and most of the time this will be the case, does it pay to check for this case, and if so not execute the filter at all? or is the __in lookup efficient enough that even if user_id_list contains all users it'll be more efficient to run the filter. Also, do I need to worry about making the resulting test_attempts distinct()? or they can't possible turn up duplicates with the structure of my queryset?

EDIT: For anyone who's interested in looking at the raw SQL query, it looks like this without the filter:

SELECT "mc_grades_testattempt"."id", "mc_grades_testattempt"."date", 
"mc_grades_testattempt"."test_id", "mc_grades_testattempt"."student_id" FROM 
"mc_grades_testattempt" WHERE "mc_grades_testattempt"."test_id" = 1

and this with the filter:

SELECT "mc_grades_testattempt"."id", "mc_grades_testattempt"."date", 
"mc_grades_testattempt"."test_id", "mc_grades_testattempt"."student_id" FROM 
"mc_grades_testattempt" INNER JOIN "mc_grades_userprofile" ON 
("mc_grades_testattempt"."student_id" = "mc_grades_userprofile"."id") WHERE 
("mc_grades_testattempt"."test_id" = 1  AND "mc_grades_userprofile"."user_id" IN (1, 2, 3))

note that the array (1,2,3) is just an example

1. Short answer is – benchmark. Test it in different situations and measure the load. It will be the best answer.

2. There can't be duplicates here.

3. Is it really a problem to check for two situalions? Here's the hypotetic code:

   def average_score(self, user_id_list=None):
       qset = self.test_attempts.all()
       if user_id_list is not None:
           qset = qset.filter(profile__id__in=user_id_list)
       scores = [x.score() for x in qset]
       # and compute the average
   

4. I don't know what does score method do, but can't you compute the average at DB level? It will give you much more noticable perfomance boost.

5. And don't forget about caching.

I have a model (Node) which is ordered by date in the admin, so latest nodes are shown first. This is fine.

The same model (Node) is referenced by another model (Device). When editing a device there is a list of nodes (in an HTML select) which is also ordered by date. I would like this select to be ordered by name and not by date.

Is it possible to have two different ordering methods, one for the list of objects and one for the select box?

Thanks.

The easiest thing would be to override the formfield_for_foreignkey method in the ModelAdmin for Device, something like

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    if db_field.name == 'node':
        kwargs['queryset'] = Node.objects.order_by('name')
    return super(DeviceAdmin, self).formfield_for_foreignkey(db_field, request, **kwargs)

(I'm assuming a fair amount here. Hopefully it's clear!)

Similarly there's formfield_for_manytomany.

I dont know if at all i make any sense, but this popped up in my mind. Can we use the 2gb free hosting of dropbox to put our django app over there and do some hacks to run our app?

The point you need to understand is: can you run django without djagno installed? Can django be installed on a usb-drive? Dropbox is but a cloud storage service NOT cloud hosting service. To store you do not require RAM and processors while for hosting you do.

Hence the answer is NO. If you are okay with configuring apache etc. you can go for VPS. If not try some managed servers. If you dont want to spend much and are looking at free hosting solutions for django a few of them are:

kodingen.com

shellmix.com

http://0x2a-dc.com/index.php?name=shop&cat=6

http://www.heliohost.org/home/

http://bitnami.org/cloud

https://www.alwaysdata.com/

more data: http://freedjangohosting.com/

http://djangohosting.com/

https://code.djangoproject.com/wiki/DjangoFriendlyWebHosts

I realize there are a ton of questions on this subject, and I have been researching this for a couple days now. I want to make sure my question is as specific as possible since I have yet to gain a full understanding of the best approach.

Currently I have a developed django site, with the web client communicating probably about 95% via a django-piston json REST api. The other 5% is some remaning login functionality that still goes through POST forms with CSRF protection. Ideally I would like to move the remainder also into the REST api.

I am at the point now where I need to figure out the best recommended solution for securing both the web client and the mobile client (app yet to be developed) in a reusable and happily co-existing fashion. I have read many posts ultimately recommending OAuth2 (and https) for the mobile side, but I am still confused about how to go about setting up the web client security. I am also grasping at understanding the OAuth2 aspect and whether I can use the 2-legged form. As it stands, the web client is django authenticated. Technically the jsonp functionality is still active in piston, so I would think anyone could use the api from a 3rd party app as long as their web session has the auth cookies?

Summary of the usage of my api:

1. The API is a completely private interface to the server app
2. It would be ideal if the API could not be widely reused by 3rd party web client mashups.
3. The data is not necc sensitive. Its just a social-type site with the most personal information being the basic user profile stuff like emails, addresses, etc.

Summary of my questions:

1. Is OAuth2 the best recommended approach to securing the mobile apps access? Does it have anything to do with the web client aspect? And if OAuth2 is recommended, should it be an application-wide key that is versioned with the app releases?
2. Should the web client use CSRF that is passed over ajax, and just disable jsonp to ensure its always same origin? Basically, am I treating the web client security separately?
3. How should I go about organizing the urls/app instances/subdomains or whatever is recommended to maintain the web vs mobile security? Do I just need separate url prefixes, one for mobile that uses different rules?

I am looking for django-piston specific recommendations to solving these problems. I have already branched my project and started to play with this forked version of piston: https://bitbucket.org/jespern/django-piston-oauth2

One idea I had was to create a piston resource that first checks if its same-origin and then only enforces the django auth, otherwise it enforces oauth2, but I am not sure if this is even appropriate.

Update 1/1/2012

From the info that Spike provided, I started working with piston-oauth2. I ended up creating a fork of that to add some fixes for nonrel django (mongodb) and I forked someones example to also use oauth2 and piston:

https://bitbucket.org/justinfx/django-piston-oauth2-nonrel-example

Now its just a matter of me really hooking this up to my own project and getting that working. But these tests all work great.

I'm all for OAuth2, so I'll reply based on that solution.

Is OAuth2 the best recommended approach to securing the mobile apps access? Does it have anything to do with the web client aspect? And if OAuth2 is recommended, should it be an application-wide key that is versioned with the app releases?

Yes, OAuth2 widely regarded as the recommended approach at the moment. It's far easier than OAuth1. I'd recommend actually reading the spec instead of blog posts about the spec as the spec itself is very clearly written. Beyond the spec, it's useful to look at established implementations of it like Facebook's and Foursquare's since they don't follow the spec in every way, but make some modifications to be more practical and easy to use.

As for versioning the releases, from a dogmatic REST perspective this is frowned upon. However, from a more pragmatic perspective, this is extremely common practice and makes life much simpler for both the API developers and the clients. I'd recommend reading the Apigee blog, as they have lots of posts about topics like versioning.

Should the web client use CSRF that is passed over ajax, and just disable jsonp to ensure its always same origin? Basically, am I treating the web client security separately?

If you go with a full oauth2 solution, you'll want to enable cross-site api requests. To deny apps you don't know, you can just add checks for that when you look at the access_tokens being passed in. Here's some reading about the different options you have:

http://blog.apigee.com/detail/crossing_the_streams_handling_cross-site_api_requests/

How should I go about organizing the urls/app instances/subdomains or whatever is recommended to maintain the web vs mobile security? Do I just need separate url prefixes, one for mobile that uses different rules?

Just decide what works for you. Lots of people have their mobile site at "m.mysite.com" or "mobile.mysite.com" these days. This decision isn't really related to the whole authentication discussion if you go with a full OAuth2 implementation.

I am looking for django-piston specific recommendations to solving these problems. I have already branched my project and started to play with this forked version of piston: https://bitbucket.org/jespern/django-piston-oauth2

I'm not familiar with this, as I use tastypie. If it doesn't work well for you, there is an excellent Django OAuth2 standalone server that I've used:

https://github.com/hiidef/oauth2app

Here is what I'm trying to do:

I want to be able to finish the "OAuth token dance" and gain an access token so I can then use that to connect to googles IMAP api for a user.

Here are my problems:

I feel like I've tried almost everything. I've tried using the GDClient, GDataService, and Django Social Auth OAuth clients but I still come up empty. I've ran into scenarios where google would build the authorization url but the initial request token returns empty, preventing me from being able to gain an access token (that happened while I was using the GDataClient).

Here is what I would like to have:

A complete resource/tutorial/sample of how to gain an access token from google using OAuth or OAuth2.0. I'm pretty sure I can work the IMAP API out on my own. I am using the django framework, but I'm willing to switch from that if there is a better option.

Note: google does have good information in their documentation but i feel like it is too scrambled at the moment. I found myself mixing OAuth and OAuth2.0 because of it.

Any help is greatly appreciated. I will personally consider anyone who has pulled off google authentication using OAuth or OAuth2.0 awesome, because you would have to be if you used the documentation I have come across.

Sample

def index(request):

    scopes = ['https://docs.google.com/feeds/','https://www.google.com/calendar/feeds/']  

    client = gdata.docs.client.DocsClient(source='Trinity-EmailManager-v1')
    client.ssl = True
    client.http_client.debug = True

    oauth_callback_url = settings.GOOGLE_CALLBACK_URL
    request_token = client.GetOAuthToken(
        scopes, oauth_callback_url, settings.GOOGLE_CONSUMER_KEY, consumer_secret=settings.GOOGLE_CONSUMER_SECRET)

return HttpResponse(request_token)

I haven't done it in django, but I did patch OpenID using Google exclusively into a Pylons project. I just used Python OpenID. Its been a few months since I set that up, but I seem to remember the Google Documentation and the Documentation for the OpenID package to be reasonable to get me through it, although it took quite a bit to get over a few of the hurdles. If I remember correctly, the biggest problem was getting Google to give me back the data I wanted, which was in setting up the AX Requests properly.

In the model I was using, you set up the Authentication request, present a link to the user, and it calls back to your page. The consumer object from the OpenID package was able to take the redirect from Google and parse it just fine then.

Can you be a little more specific about what is going on? You mention that the request token was empty, which I handle by assuming an issue with the authentication. Are you getting no response at all?

I have a site that requires the ability for a logged in admin to push a staging database to a live database. The first thing it does is dump the sql and push to the target database. This works fine, but when I go to rsync the folders containing the uploaded material, I get an error. This ONLY occurs when the script is called from within the view, not from the command line or python shell. Here is the function:

def copy_media(self, origin_folder, target_folder):
    command_string = "rsync -a %s %s" % (origin_folder, target_folder)
    return_code = subprocess.call(command_string, shell=True)
    return return_code

The return code is "12" when it errors. My best guess is that since there's a considerable delay before the script finishes executing, the view doesn't know how to properly wait for it to end. The other guess I had was that the paths somehow get screwed up from within the view.

Check permissions of your server, it may be different user/permissions from when you use the command line and thus not be able to perform that command.