Interesting socket behavior exhibited while processing Django view

From Devipedia

Jump to: navigation, search

I needed to have a Django view connect to another local port while processing a request. I had written a Twisted based TCP handler for this purpose and had been using it successfully from the command line. My goal was to wrap the entire interaction in a function, like so:

def get_info_from_socket(msg):
     Hide the fact that process is asynchronous.  Talk to the port, get the answer, 
     or return an error.

Here's the twisted code to handle the communication with the socket.


from twisted.internet.protocol import Protocol, ClientFactory
from twisted.internet import reactor
from twisted.internet import defer

class Send(Protocol):

    retval = ''

    def __init__(self, msg):
        self.msg = msg

    def connectionMade(self):

        l = len(self.msg)
        fmsg = "%04d%s" % (l,self.msg)
        print "sending..." 

    def dataReceived(self, data):
        self.retval += data

    def connectionLost(self, reason):
        print "connection lost"

class SendClientFactory(ClientFactory):
    def __init__(self, msg ):
        self.msg = msg

    def buildProtocol(self, addr):
        print 'Connected.'
        self.s = Send(self.msg)
        return self.s

    def clientConnectionLost(self, connector, reason):
        self.retval = self.s.retval

    def clientConnectionFailed(self, connector, reason):
        self.retval = 'Client Connection Failed: %s' % reason

def iRemoteRunner(msg):
    f = SendClientFactory(msg)
    reactor.connectTCP('localhost', 5956, f)
    #  This code is here to show you what NOT to do.  Don't copy and paste for use elsewhere
    #  unless you understand why this is a mistake.
    return f.retval

When calling iRemoteRunner from main (command line), this worked absolutely as expected, returning a string of return values. But when invoking from within a Django view method (Django running under Apache via modwsgi), the return value was an empty string. This was not what I expected.

The Twisted reactor typically runs until it is explicitly stopped with the reactor.stop() method. The code above is flawed because I assumed the run() method would block until the answer was returned. So much for assumptions. The only reason that f.retval contained the proper result was because the interaction with the external port had already completed by the time the interpreter got to it.

As I googled for the best pattern for creating a synchronous wrapper around a Twisted network exchange, I found that the answer was: Don't use Twisted if you want synchronous behavior.

Follow this thread for more info. There are many others like it:

So aside from using the wrong networking framework, why is it that I was not getting the answer? Why, when running from the command line, was there enough processing time to retrieve the answer, but when running under the webserver there was not?

I replaced the Twisted calls with calls to the Socket library:

import socket

def iRemoteRunner(msg):
    s = socket.socket( socket.AF_INET, socket.SOCK_STREAM)
    s.connect(('localhost', 5956))
    s.send('%s%s' % (len(msg),msg))
    retval = s.recv( 300 )

When calling from the command line, this small snippet of code provided the same net effect (an appropriate response from the server running on port 5956). However, when this method was called from inside the Django view, the process would hang.

In both cases it seems that the server could not pull a response after opening the socket to the port. In the first case, because Twisted is asynchronous, my result was an empty string because the response never arrived. In the second case, because s.recv blocks, the process hung while waiting for a response.

I decided to try another high-level library to see if the result would be the same. This time I used telnet.

from telnetlib import Telnet

def iRemoteRunner(msg):   
    tn = Telnet('localhost', 5956)
    tn.write("%04d%s" % (len(msg), msg))
    return tn.read_all()

This solution not only proved to be the most concise but also the only one to work when invoked through Apache/Django.

I suspect that there is a library conflict, perhaps at the socket library level, that prevented the Twisted and socket based solutions from working.

If you have any insights, please leave a comment here:

Personal tools