Search Results

Search found 209 results on 9 pages for 'twisted'.

Page 2/9 | < Previous Page | 1 2 3 4 5 6 7 8 9  | Next Page >

• Writing a blocking wrapper around twisted's IRC client

  - by Andrey Fedorov

  I'm trying to write a dead-simple interface for an IRC library, like so: import simpleirc connection = simpleirc.Connect('irc.freenode.net', 6667) channel = connection.join('foo') find_command = re.compile(r'google ([a-z]+)').findall for msg in channel: for t in find_command(msg): channel.say("http://google.com/search?q=%s" % t) Working from their example, I'm running into trouble (code is a bit lengthy, so I pasted it here). Since the call to channel.__next__ needs to be returned when the callback <IRCClient instance>.privmsg is called, there doesn't seem to be a clean option. Using exceptions or threads seems like the wrong thing here, is there a simpler (blocking?) way of using twisted that would make this possible?

  Read the article

• How different protocols interact with eachother in Twisted

  - by stsupermouse

  The scenario I want two different protocols interact with each other is as below: A and B is two different protocols. First A will interact with the server and retrieve some values. Only after A finishes retrieving the values , B will start to interact with the server. Now my problem is that is there an elegant way to initial B when A retrieves the values. Currently I just initial B in A's data process function. But i don't think that this is an elegant way. What I mean an elegant way is that the initialization of B is done by a flow controller or something like that, but not another protocol. Is there an elegant way? such using defered or any other things. I'm just new to twisted, not knowing very much about defered.... Thank you very much!

  Read the article

• chatbot using twisted and wokkel

  - by dmitriy k.

  I am writing a chatbot using Twisted and wokkel and everything seems to be working except that bot periodically logs off. To temporarily fix that I set presence to available on every connection initialized. Does anyone know how to prevent going offline? (I assume if i keep sending available presence every minute or so bot wont go offline but that just seems too wasteful.) Suggestions anyone? Here is the presence code: class BotPresenceClientProtocol(PresenceClientProtocol): def connectionInitialized(self): PresenceClientProtocol.connectionInitialized(self) self.available(statuses={None: 'Here'}) def subscribeReceived(self, entity): self.subscribed(entity) self.available(statuses={None: 'Here'}) def unsubscribeReceived(self, entity): self.unsubscribed(entity) Thanks in advance.

  Read the article

• Twisted Spread: How to authenticate each RPC with digital signature

  - by kronat

  I have remote objects which talk each others with RPCs, using Twisted Spread. I want that objects authenticate messages, before using them, with digital signatures, but I don't know where to start to implement this. In my head, the Root object must have a public/private key pair, and the Client too. When a message is sent, a digital signature of the hash is added, and when it is received, the signature is checked. Is the Protocol part where I need to add these adds and checks? Thank you

  Read the article

• Sending buffered images between Java client and Twisted Python socket server

  - by PattimusPrime

  I have a server-side function that draws an image with the Python Imaging Library. The Java client requests an image, which is returned via socket and converted to a BufferedImage. I prefix the data with the size of the image to be sent, followed by a CR. I then read this number of bytes from the socket input stream and attempt to use ImageIO to convert to a BufferedImage. In abbreviated code for the client: public String writeAndReadSocket(String request) { // Write text to the socket BufferedWriter bufferedWriter = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream())); bufferedWriter.write(request); bufferedWriter.flush(); // Read text from the socket BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(socket.getInputStream())); // Read the prefixed size int size = Integer.parseInt(bufferedReader.readLine()); // Get that many bytes from the stream char[] buf = new char[size]; bufferedReader.read(buf, 0, size); return new String(buf); } public BufferedImage stringToBufferedImage(String imageBytes) { return ImageIO.read(new ByteArrayInputStream(s.getBytes())); } and the server: # Twisted server code here # The analog of the following method is called with the proper client # request and the result is written to the socket. def worker_thread(): img = draw_function() buf = StringIO.StringIO() img.save(buf, format="PNG") img_string = buf.getvalue() return "%i\r%s" % (sys.getsizeof(img_string), img_string) This works for sending and receiving Strings, but image conversion (usually) fails. I'm trying to understand why the images are not being read properly. My best guess is that the client is not reading the proper number of bytes, but I honestly don't know why that would be the case. Side notes: I realize that the char[]-to-String-to-bytes-to-BufferedImage Java logic is roundabout, but reading the bytestream directly produces the same errors. I have a version of this working where the client socket isn't persistent, ie. the request is processed and the connection is dropped. That version works fine, as I don't need to care about the image size, but I want to learn why the proposed approach doesn't work.

  Read the article

• Twisted: why is it that passing a deferred callback to a deferred thread makes the thread blocking a

  - by surtyaarthoughts

  I unsuccessfully tried using txredis (the non blocking twisted api for redis) for a persisting message queue I'm trying to set up with a scrapy project I am working on. I found that although the client was not blocking, it became much slower than it could have been because what should have been one event in the reactor loop was split up into thousands of steps. So instead, I tried making use of redis-py (the regular blocking twisted api) and wrapping the call in a deferred thread. It works great, however I want to perform an inner deferred when I make a call to redis as I would like to set up connection pooling in attempts to speed things up further. Below is my interpretation of some sample code taken from the twisted docs for a deferred thread to illustrate my use case: #!/usr/bin/env python from twisted.internet import reactor,threads from twisted.internet.task import LoopingCall import time def main_loop(): print 'doing stuff in main loop.. do not block me!' def aBlockingRedisCall(): print 'doing lookup... this may take a while' time.sleep(10) return 'results from redis' def result(res): print res def main(): lc = LoopingCall(main_loop) lc.start(2) d = threads.deferToThread(aBlockingRedisCall) d.addCallback(result) reactor.run() if __name__=='__main__': main() And here is my alteration for connection pooling that makes the code in the deferred thread blocking : #!/usr/bin/env python from twisted.internet import reactor,defer from twisted.internet.task import LoopingCall import time def main_loop(): print 'doing stuff in main loop.. do not block me!' def aBlockingRedisCall(x): if x<5: #all connections are busy, try later print '%s is less than 5, get a redis client later' % x x+=1 d = defer.Deferred() d.addCallback(aBlockingRedisCall) reactor.callLater(1.0,d.callback,x) return d else: print 'got a redis client; doing lookup.. this may take a while' time.sleep(10) # this is now blocking.. any ideas? d = defer.Deferred() d.addCallback(gotFinalResult) d.callback(x) return d def gotFinalResult(x): return 'final result is %s' % x def result(res): print res def aBlockingMethod(): print 'going to sleep...' time.sleep(10) print 'woke up' def main(): lc = LoopingCall(main_loop) lc.start(2) d = defer.Deferred() d.addCallback(aBlockingRedisCall) d.addCallback(result) reactor.callInThread(d.callback, 1) reactor.run() if __name__=='__main__': main() So my question is, does anyone know why my alteration causes the deferred thread to be blocking and/or can anyone suggest a better solution?

  Read the article

• Python (Twisted) - reading from fifo and sending read data to multiple protocols

  - by SpankMe

  Hi, Im trying to write some kind of multi protocol bot (jabber/irc) that would read messages from fifo file (one liners mostly) and then send them to irc channel and jabber contacts. So far, I managed to create two factories to connect to jabber and irc, and they seem to be working. However, I've problem with reading the fifo file - I have no idea how to read it in a loop (open file, read line, close file, jump to open file and so on) outside of reactor loop to get the data I need to send, and then get that data to reactor loop for sending in both protocols. I've been looking for information on how to do it in best way, but Im totally lost in the dark. Any suggestion/help would be highly appreciated. Thanks in advance!

  Read the article

• Twisted - how to create multi protocol process and send the data between the protocols

  - by SpankMe

  Hey, Im trying to write a program that would be listening for data (simple text messages) on some port (say tcp 6666) and then pass them to one or more different protocols - irc, xmpp and so on. I've tried many approaches and digged the Internet, but I cant find easy and working solution for such task. The code I am currently fighting with is here: http://pastebin.com/ri7caXih I would like to know how to from object like: ircf = ircFactory('asdfasdf', '#asdf666') get access to self protocol methods, because this: self.protocol.dupa1(msg) returns error about self not being passed to active protocol object. Or maybe there is other, better, easier and more kosher way to create single reactor with multiple protocols and have actions triggeres when a message arrives on any of them, and then pass that message to other protocols for handling/processing/sending? Any help will be highly appreciated!

  Read the article

• twisted reactor stops too early

  - by pygabriel

  I'm doing a batch script to connect to a tcp server and then exiting. My problem is that I can't stop the reactor, for example: cmd = raw_input("Command: ") # custom factory, the protocol just send a line reactor.connectTCP(HOST,PORT, CommandClientFactory(cmd) d = defer.Deferred() d.addCallback(lambda x: reactor.stop()) reactor.callWhenRunning(d.callback,None) reactor.run() In this code the reactor stops before that the tcp connection is done and the cmd is passed. How can I stop the reactor after that all the operation are finished?

  Read the article

• how do twisted/tornado et cetera work

  - by aharon

  I understand that they work in some way distinct from making a thread per user. How exactly does that work? (Does 'non-blocking' have something to do with it?)

  Read the article

• Sending message from one server to another in Twisted

  - by Casey Patton

  I've implemented my servers in the following way: def makeServer(application, port): factory = protocol.ServerFactory() factory.protocol = MyChat factory.clients = [] internet.TCPServer(port, factory).setServiceParent(application) application = service.Application("chatserver") server1 = makeServer(application, port=1025) server2 = makeServer(application, port=1026) server3 = makeServer(application, port=1027) Note that MyChat is an event handling class that has a "receiveMessage" action: def lineReceived(self, line): print "received", repr(line) for c in self.factory.clients: c.transport.write(message + '\n') I want server1 to be able to pass messages to server2. Rather, I want server1 to be treated as a client of server2. If server1 receives the message "hi" then I want it to send that same exact message to server2. How can I accomplish this?

  Read the article

• twisted .loseConnection does not immediately lose connection?

  - by Claudiu

  I have a server with a few clients connected to it. When CTRL+C is hit (that is, reactor starts shutting down), I want to close all my connections, wait until they are cleanly closed, and then stop. I do this by going through the connected clients' transports and calling .loseConnection(). On the ones that are connected locally, they immediately disconnect. However, on one that is connected through the internet, the connection is not immediately lost. Communication stops - and closing the client program no longer even tells the server that the connection has died, although it does before calling .loseConnection() - but the connection is not deemed 'lost' until a few minutes later after I send a few heartbeat requests from the server. I understand that if a connection dies, there's no way for the server to know unless it tries to send some data. But if I specifically ask for a connection to be closed, why does it not just close/disconnect immediately? Am I calling the wrong function?

  Read the article

• Twisted Matrix and telnet server implementation

  - by ypercube

  I have a project which is essentially a game server where users connect and send text commands via telnet. The code is in C and really old and unmodular and has several bugs and missing features. The main function alone is half the code. I came to the conclusion that rewriting it in TwistedMachine could actually result in faster completement, besides other benefits. So, here is the questions: What packages and modules I should use? I see a "telnet" module inside "protocols" package. I also see "cronch" package with "ssh" and another "telnet" module. I'm a complete novice regarding Python.

  Read the article

• twisted DeferredList parallel Image Download

  - by bell007

  This is code: http://www.dpaste.de/Ij0S/ 1,if there is a erorr (networking erorr,or Unhandled error in Deferred), the code stop. I need this code running until all urls finish request. (May be the parallel function not work? ) 2,when I write image to local filesystem, if meet erorr, there images may not complete. Thanks!

  Read the article

• What should I name instances of a twisted.internet.defer.Deferred?

  - by slacy

  I'm writing code using Twisted, and having trouble coming up with a sensible variable name for my twisted internet deferred's. Here are my candidates: d : Too generic, too short, violates pylint rule C0103. def : Conflicts with function defintion builtin. defer : Conflicts with module twisted.internet.defer deferred : OK but pretty long cb : Still too short, violates pylint C0103, conflicts with many callback method names. cback : Too Weird? callback : Conflicts with method Deferred.callback() I'm looking for other suggestions. It seems like most of the Twisted example code uses "d" which is fine for simple invocations, but when you're passing Deferred's around to methods and storing them as member variables, it's really far too descriptive.

  Read the article

• reactor not working when reactor.run is not called in the main thread and installSignalHandlers=Fals

  - by Kalmi

  I'm trying to answer the following question out of personal interest: What is the fastest way to send 100,000 HTTP requests in Python? And this is what I have came up so far, but I'm experiencing something very stange. When installSignalHandlers is True, it just hangs. I can see that the DelayedCall instances are in reactor._newTimedCalls, but processResponse never gets called. When installSignalHandlers is False, it throws an error and works. from twisted.internet import reactor from twisted.web.client import Agent from threading import Semaphore, Thread import time concurrent = 100 s = Semaphore(concurrent) reactor.suggestThreadPoolSize(concurrent) t=Thread( target=reactor.run, kwargs={'installSignalHandlers':True}) t.daemon=True t.start() agent = Agent(reactor) def processResponse(response,url): print response.code, url s.release() def processError(response,url): print "error", url s.release() def addTask(url): req = agent.request('HEAD', url) req.addCallback(processResponse, url) req.addErrback(processError, url) for url in open('urllist.txt'): addTask(url.strip()) s.acquire() while s._Semaphore__value!=concurrent: time.sleep(0.1) reactor.stop() And here is the error that it throws when installSignalHandlers is True: (Note: This is the expected behaviour! The question is why it doesn't work when installSignalHandlers is False.) Traceback (most recent call last): File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 396, in fireEvent DeferredList(beforeResults).addCallback(self._continueFiring) File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 224, in addCallback callbackKeywords=kw) File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 213, in addCallbacks self._runCallbacks() File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 371, in _runCallbacks self.result = callback(self.result, *args, **kw) --- <exception caught here> --- File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 409, in _continueFiring callable(*args, **kwargs) File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 1165, in _reallyStartRunning self._handleSignals() File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 1105, in _handleSignals signal.signal(signal.SIGINT, self.sigInt) exceptions.ValueError: signal only works in main thread What am I doing wrong and what is the right way? I'm new to twisted.

  Read the article

• Would Python's Twisted library be the best case for an observer type pattern?

  - by beagleguy

  hi all, I'm developing a system where a queue will be filled with millions of items I need a process that reads items from the queue constantly and then sends those items out to registered clients. I'm thinking about using twisted for this, having the queue reader be a twisted server listening on a tcp port then clients can connect on that port and when an item is pulled from the queue the server writes it out to all the clients. Does that sound like something that twisted would be ideal for? Does anyone know of any sample code out there that may do something similar? thanks

  Read the article

• Twisted + SQLAlchemy and the best way to do it.

  - by Khorkrak

  So I'm writing yet another Twisted based daemon. It'll have an xmlrpc interface as usual so I can easily communicate with it and have other processes interchange data with it as needed. This daemon needs to access a database. We've been using SQL Alchemy in place of hard coding SQL strings for our latest projects - those mostly done for web apps in Pylons. We'd like to do the same for this app and re-use library code that makes use of SQL Alchemy. So what to do? Well of course since that library was written for use in a Pylons app it's all the straight-forward blocking style code that everyone is accustomed to and all of the non-blocking is magically handled by Pylons via threading, thread locals, scoped sessions and so on. So now for Twisted I guess I'm a bit stuck. I could: Just write the sql I need directly if it's minimal and use the dbapi pool in twisted to do runInteractions etc when I need to hit the db. Use the objects and inherently blocking methods in our library and block now and then in my Twisted daemon. Bah. Use sAsync which was last updated in 2008 and kind of reuse the models we have defined already but not really and it does address code that needs to work in Pylons either. Does that even work with the latest version SQL Alchemy? Who knows. That project looked great though - why was it apparently abandoned? Spawn a separate subprocess and have it deal with the library code and all it's blocking, the results being returned back to my daemon when ready as objects marshalled via YAML over xmlrpc. Use deferToThread and then expunge the objects returned having made sure to do eager loads so that I have all my stuff that I might need. Seems kind of ugha to me. I'm also stuck using Python 2.5.4 atm so no 2.6 yet and I don't think I can just do an import from future to get access to the cool new multiprocessing module stuff in there. That's OK though I guess as we've got dealing with interprocess communication down pretty well. So I'm leaning towards option 4 mostly as that would avoid the mortal sin of logic duplication with option 1 while also staying the heck away from threads. Any better ideas?

  Read the article

• Queue remote calls to a Python Twisted perspective broker?

  - by agartland

  The strength of Twisted (for python) is its asynchronous framework (I think). I've written an image processing server that takes requests via Perspective Broker. It works great as long as I feed it less than a couple hundred images at a time. However, sometimes it gets spiked with hundreds of images at virtually the same time. Because it tries to process them all concurrently the server crashes. As a solution I'd like to queue up the remote_calls on the server so that it only processes ~100 images at a time. It seems like this might be something that Twisted already does, but I can't seem to find it. Any ideas on how to start implementing this? A point in the right direction? Thanks!

  Read the article

• Twisted: how-to bind a server to a specified IP address? (solved)

  - by daccle

  I want to have a twisted service (started via twistd) which listens to TCP/POST request on a specified port on a specified IP address. By now I have a twisted application which listens to port 8040 on localhost. It is running fine, but I want it to only listen to a certain IP address, say 10.0.0.78. How-to manage that? This is a snippet of my code: application = service.Application('SMS_Inbound') smsInbound = resource.Resource() smsInbound.putChild('75sms_inbound',ReceiveSMS(application)) smsInboundServer = internet.TCPServer(8001, webserver.Site(smsInbound)) smsInboundServer.setName("SMS Handling") smsInboundServer.setServiceParent(application)

  Read the article

• Eventlet or gevent or Stackless + Twisted, Pylons, Django and SQL Alchemy

  - by Khorkrak

  We're using Twisted extensively for apps requiring a great deal of asynchronous io. There are some cases where stuff is cpu bound instead and for that we spawn a pool of processes to do the work and have a system for managing these across multiple servers as well - all done in Twisted. Works great. The problem is that it's hard to bring new team members up to speed. Writing asynchronous code in Twisted requires a near vertical learning curve. It's as if humans just don't think that way naturally. We're considering a mixed approach perhaps. Maybe do the xmlrpc server part and process management in Twisted still but the other stuff in code that at least looks synchronous while not being as such. Then again I like explicit over implicit so hmmm. Anyway onto greenlets - how well does that stuff work? So there's Stackless and as you can see from my Gallentean avatar I'm well aware of the tremendous success in it's use for CCP's flagship EVE Online game first hand. What about Eventlet or gevent? Well for now only Eventlet works with Twisted. However gevent claims to be faster since it's not a pure python implementation it instead uses libevent. It also has fewer idiosyncrasies and defects supposedly. The documentation there is minimal in comparison to Eventlet and it's maintained by 1 guy as far as I can tell. This makes me leery but all great projects start this way so... Then there's PyPy - I haven't even finished reading about that one yet - just saw it in this thread: Drawbacks of Stackless. So confusing - I'm wondering what the heck to do - sounds like Eventlet is probably the best bet but is it really stable enough? Anyone out there have any experience with it? Should we go with Stackless instead as it's been around and is proven technology - just like Twisted is as well - and they do work together nicely. But still I hate having to have a separate version of Python to do this. what to do.... This somewhat obnoxious blog entry hit the nail on the head for me though: Asynchronous IO for Grownups We're stuck using MySQL as well - I never knew how great PostgreSQL was until having had to work on a production OLTP system in MySQL instead - but that's another story. But if that monkey patch thing really works then wow. Just wow.

  Read the article

• Caveats of select/poll vs. epoll reactors in Twisted

  - by David

  Everything I've read and experienced ( Tornado based apps ) leads me to believe that ePoll is a natural replacement for Select and Poll based networking, especially with Twisted. Which makes me paranoid, its pretty rare for a better technique or methodology not to come with a price. Reading a couple dozen comparisons between epoll and alternatives shows that epoll is clearly the champion for speed and scalability, specifically that it scales in a linear fashion which is fantastic. That said, what about processor and memory utilization, is epoll still the champ?

  Read the article

• Twisted Python getPage

  - by David Dixon II

  I tried to get support on this but I am TOTALLY confused. Here's my code: from twisted.internet import reactor from twisted.web.client import getPage from twisted.web.error import Error from twisted.internet.defer import DeferredList from sys import argv class GrabPage: def __init__(self, page): self.page = page def start(self, *args): if args == (): # We apparently don't need authentication for this d1 = getPage(self.page) else: if len(args) == 2: # We have our login information d1 = getPage(self.page, headers={"Authorization": " ".join(args)}) else: raise Exception('Missing parameters') d1.addCallback(self.pageCallback) dl = DeferredList([d1]) d1.addErrback(self.errorHandler) dl.addCallback(self.listCallback) def errorHandler(self,result): # Bad thingy! pass def pageCallback(self, result): return result def listCallback(self, result): print result a = GrabPage('http://www.google.com') data = a.start() # Not the HTML I wish to get the HTML out which is given to pageCallback when start() is called. This has been a pita for me. Ty! And sorry for my sucky coding.

  Read the article

• What is the Lua equivalent of Twisted Python, Eventmachine for Ruby, NIO for Java, etc.?

  - by z8000

  What is the Lua equivalent of Twisted Python, Eventmachine for Ruby, NIO for Java, etc.? If there is none, how does one do non-blocking event-based network I/O? In general, how does one solve the C10K problem in Lua? Thanks!

  Read the article

• Sending data from one Protocol to another Protocol in Twisted?

  - by veb

  Hi! One of my protocols is connected to a server, and with the output of that I'd like to send it to the other protocol. I need to access the 'msg' method in ClassA from ClassB but I keep getting: exceptions.AttributeError: 'NoneType' object has no attribute 'write' Actual code: http://pastebin.com/MQPhduSY Any ideas please? :-)

  Read the article

< Previous Page | 1 2 3 4 5 6 7 8 9  | Next Page >