Search Results

Search found 15224 results on 609 pages for 'parallel python'.

Page 155/609 | < Previous Page | 151 152 153 154 155 156 157 158 159 160 161 162  | Next Page >

  • Memory problems while code is running (Python, Networkx)

    - by MIN SU PARK
    I made a code for generate a graph with 379613734 edges. But the code couldn't be finished because of memory. It takes about 97% of server memory when it go through 62 million lines. So I killed it. Do you have any idea to solve this problem? My code is like this: import os, sys import time import networkx as nx G = nx.Graph() ptime = time.time() j = 1 for line in open("./US_Health_Links.txt", 'r'): #for line in open("./test_network.txt", 'r'): follower = line.strip().split()[0] followee = line.strip().split()[1] G.add_edge(follower, followee) if j%1000000 == 0: print j*1.0/1000000, "million lines done", time.time() - ptime ptime = time.time() j += 1 DG = G.to_directed() # P = nx.path_graph(DG) Nn_G = G.number_of_nodes() N_CC = nx.number_connected_components(G) LCC = nx.connected_component_subgraphs(G)[0] n_LCC = LCC.nodes() Nn_LCC = LCC.number_of_nodes() inDegree = DG.in_degree() outDegree = DG.out_degree() Density = nx.density(G) # Diameter = nx.diameter(G) # Centrality = nx.betweenness_centrality(PDG, normalized=True, weighted_edges=False) # Clustering = nx.average_clustering(G) print "number of nodes in G\t" + str(Nn_G) + '\n' + "number of CC in G\t" + str(N_CC) + '\n' + "number of nodes in LCC\t" + str(Nn_LCC) + '\n' + "Density of G\t" + str(Density) + '\n' # sys.exit() # j += 1 The edge data is like this: 1000 1001 1000245 1020191 1000 10267352 1000653 10957902 1000 11039092 1000 1118691 10346 11882 1000 1228281 1000 1247041 1000 12965332 121340 13027572 1000 13075072 1000 13183162 1000 13250162 1214 13326292 1000 13452672 1000 13844892 1000 14061830 12340 1406481 1000 14134703 1000 14216951 1000 14254402 12134 14258044 1000 14270791 1000 14278978 12134 14313332 1000 14392970 1000 14441172 1000 14497568 1000 14502775 1000 14595635 1000 14620544 1000 14632615 10234 14680596 1000 14956164 10230 14998341 112000 15132211 1000 15145450 100 15285998 1000 15288974 1000 15300187 1000 1532061 1000 15326300 Lastly, is there anybody who has an experience to analyze Twitter link data? It's quite hard to me to take a directed graph and calculate average/median indegree and outdegree of nodes. Any help or idea?

    Read the article

  • Python __subclasses__() not listing subclasses

    - by Mridang Agarwalla
    I cant seem to list all derived classes using the __subclasses__() method. Here's my directory layout: import.py backends __init__.py --digger __init__.py base.py test.py --plugins plugina_plugin.py From import.py i'm calling test.py. test.py in turn iterates over all the files in the plugins directory and loads all of them. test.py looks like this: import os import sys import re sys.path.append(os.path.join(os.path.abspath(os.path.dirname(os.path.abspath( __file__ ))))) sys.path.append(os.path.join(os.path.abspath(os.path.dirname(os.path.abspath( __file__ ))), 'plugins')) from base import BasePlugin class TestImport: def __init__(self): print 'heeeeello' PLUGIN_DIRECTORY = os.path.join(os.path.abspath(os.path.dirname(os.path.abspath( __file__ ))), 'plugins') for filename in os.listdir (PLUGIN_DIRECTORY): # Ignore subfolders if os.path.isdir (os.path.join(PLUGIN_DIRECTORY, filename)): continue else: if re.match(r".*?_plugin\.py$", filename): print ('Initialising plugin : ' + filename) __import__(re.sub(r".py", r"", filename)) print ('Plugin system initialized') print BasePlugin.__subclasses__() The problem us that the __subclasses__() method doesn't show any derived classes. All plugins in the plugins directory derive from a base class in the base.py file. base.py looks like this: class BasePlugin(object): """ Base """ def __init__(self): pass plugina_plugin.py looks like this: from base import BasePlugin class PluginA(BasePlugin): """ Plugin A """ def __init__(self): pass Could anyone help me out with this? Whatm am i doing wrong? I've racked my brains over this but I cant seem to figure it out Thanks.

    Read the article

  • how to display my list with n amount on each line in Python

    - by user1786698
    im trying to display my list with 7 states on each line here is what i have so far, but it displays as one long string of all the states with quotes around each state. I forgot to mention that this is for my CS class and we havent learned iter yet so we not allowed to use it. the only hint i was given was to to turn STATE_LIST into a string then use '\n' to break it up state = str(STATE_LIST) displaystates = Text(Point(WINDOW_WIDTH/2, WINDOW_HEIGHT/2), state.split('\n')) displaystates.draw(win) and STATE_LIST looks like this STATE_VOTES = { "AL" : 9, # Alabama "AK" : 3, # Alaska "AZ" : 11, # Arizona "AR" : 6, # Arkansas "CA" : 55, # California "CO" : 9, # Colorado "CT" : 7, # Connecticut "DE" : 3, # Delaware "DC" : 3, # Washington DC "FL" : 29, # Florida "GA" : 16, # Georgia "HI" : 4, # Hawaii "ID" : 4, # Idaho "IL" : 20, # Illinois "IN" : 11, # Indiana "IA" : 6, # Iowa "KS" : 6, # Kansas "KY" : 8, # Kentucky "LA" : 8, # Louisiana "ME" : 4, # Maine "MD" : 10, # Maryland "MA" : 11, # Massachusetts "MI" : 16, # Michigan "MN" : 10, # Minnesota "MS" : 6, # Mississippi "MO" : 10, # Missouri "MT" : 3, # Montana "NE" : 5, # Nebraska "NV" : 6, # Nevada "NH" : 4, # New Hampshire "NJ" : 14, # New Jersey "NM" : 5, # New Mexico "NY" : 29, # New York "NC" : 15, # North Carolina "ND" : 3, # North Dakota "OH" : 18, # Ohio "OK" : 7, # Oklahoma "OR" : 7, # Oregon "PA" : 20, # Pennsylvania "RI" : 4, # Rhode Island "SC" : 9, # South Carolina "SD" : 3, # South Dakota "TN" : 11, # Tennessee "TX" : 38, # Texas "UT" : 6, # Utah "VT" : 3, # Vermont "VA" : 13, # Virginia "WA" : 12, # Washington "WV" : 5, # West Virginia "WI" : 10, # Wisconsin "WY" : 3 # Wyoming } STATE_LIST = sorted(list(STATE_VOTES.keys())) I am trying to get it to look somewhat like this

    Read the article

  • Exit Tks mainloop in Python?

    - by Olof
    I'm writing a slideshow program with Tkinter, but I don't know how to go to the next image without binding a key. import os, sys import Tkinter import Image, ImageTk import time root = Tkinter.Tk() w, h = root.winfo_screenwidth(), root.winfo_screenheight() root.overrideredirect(1) root.geometry("%dx%d+0+0" % (w, h)) root.focus_set() root.bind("<Escape>", lambda e: e.widget.quit()) image_path = os.path.join(os.getcwd(), 'images/') dirlist = os.listdir(image_path) for f in dirlist: try: image = Image.open(image_path+f) tkpi = ImageTk.PhotoImage(image) label_image = Tkinter.Label(root, image=tkpi) # ? label_image.place(x=0,y=0,width=w,height=h) root.mainloop(0) except IOError: pass root.destroy() I would like to add a time.sleep(10) "instead" of the root.mainloop(0) so that it would go to the next image after 10s. Now it changes when I press ESC. How can I have a timer there?

    Read the article

  • python to display the special characters

    - by Suhail
    Hi, I am facing issues with the special characters like ° and ® which represent the degreee Farenheit sign and the ® represent the registered sign, when i print the string the contains the special characters, it gives output like this: Preheat oven to 350&deg F Welcome to Lorem Ipsum Inc&reg is there a way i can output the exact characters and not their codes ? please let me know.

    Read the article

  • python regex of a date in some text, enclosed by two keywords

    - by Horace Ho
    This is Part 2 of this question and thanks very much for David's answer. What if I need to extract dates which are bounded by two keywords? Example: text = "One 09 Jun 2011 Two 10 Dec 2012 Three 15 Jan 2015 End" Case 1 bounding keyboards: "One" and "Three" Result expected: ['09 Jun 2011', '10 Dec 2012'] Case 2 bounding keyboards: "Two" and "End" Result expected: ['10 Dec 2012', '15 Jan 2015'] Thanks!

    Read the article

  • How to do cleanup reliably in python?

    - by Cheery
    I have some ctypes bindings, and for each body.New I should call body.Free. The library I'm binding doesn't have allocation routines insulated out from the rest of the code (they can be called about anywhere there), and to use couple of useful features I need to make cyclic references. I think It'd solve if I'd find a reliable way to hook destructor to an object. (weakrefs would help if they'd give me the callback just before the data is dropped. So obviously this code megafails when I put in velocity_func: class Body(object): def __init__(self, mass, inertia): self._body = body.New(mass, inertia) def __del__(self): print '__del__ %r' % self if body: body.Free(self._body) ... def set_velocity_func(self, func): self._body.contents.velocity_func = ctypes_wrapping(func) I also tried to solve it through weakrefs, with those the things seem getting just worse, just only largely more unpredictable. Even if I don't put in the velocity_func, there will appear cycles at least then when I do this: class Toy(object): def __init__(self, body): self.body.owner = self ... def collision(a, b, contacts): whatever(a.body.owner) So how to make sure Structures will get garbage collected, even if they are allocated/freed by the shared library? There's repository if you are interested about more details: http://bitbucket.org/cheery/ctypes-chipmunk/

    Read the article

  • python Requests login to website returns 403

    - by Jeff
    I'm trying to use requests to login to a website but as you can guess I'm having a problem here's the the code that I'm using import requests EMAIL = '***' PASSWORD = '***' URL = 'https://portal.bitcasa.com/login' client = requests.session(config={'verbose': sys.stderr}) login_data = {'username': EMAIL, 'password': PASSWORD,} r = client.post(URL, data=login_data, headers={"Referer": "foo"}) print r and if I print out r.text I get <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html lang="en"> <head><script type="text/javascript">var NREUMQ=NREUMQ||[];NREUMQ.push(["mark","firstbyte",new Date().getTime()])</script> <meta http-equiv="content-type" content="text/html; charset=utf-8"> <meta name="robots" content="NONE,NOARCHIVE"> <title>403 Forbidden</title> <style type="text/css"> html * { padding:0; margin:0; } body * { padding:10px 20px; } body * * { padding:0; } body { font:small sans-serif; background:#eee; } body>div { border-bottom:1px solid #ddd; } h1 { font-weight:normal; margin-bottom:.4em; } h1 span { font-size:60%; color:#666; font-weight:normal; } #info { background:#f6f6f6; } #info ul { margin: 0.5em 4em; } #info p, #summary p { padding-top:10px; } #summary { background: #ffc; } #explanation { background:#eee; border-bottom: 0px none; } </style> </head> <body> <div id="summary"> <h1>Forbidden <span>(403)</span></h1> <p>CSRF verification failed. Request aborted.</p> </div> <div id="explanation"> <p><small>More information is available with DEBUG=True.</small></p> </div> <script type="text/javascript">if(!NREUMQ.f){NREUMQ.f=function(){NREUMQ.push(["load",new Date().getTime()]);var e=document.createElement("script");e.type="text/javascript";e.src=(("http:"===document.location.protocol)?"http:":"https:")+"//"+"d1ros97qkrwjf5.cloudfront.net/42/eum/rum.js";document.body.appendChild(e);if(NREUMQ.a)NREUMQ.a();};NREUMQ.a=window.onload;window.onload=NREUMQ.f;};NREUMQ.push(["nrfj","beacon-1.newrelic.com","0e859e0620",778660,"ZAZRbUcHWBAHURFYX11MdUxbBUIKCVxKVVpSDVRWGwtfBwJeAEZRQQYdWkYUUFklQRdXZloGRHRcAlIPA0UEQ1UdE0FWVgNFEDlEDFRH",0,7,new Date().getTime(),"","","","",""])</script></body> </html> They're using a combination of django and pyramid. I've been playing around with this for about two days now but, obviously, have gotten nowhere. Thanks for your help.

    Read the article

  • How to do relative imports in Python?

    - by Joril
    Imagine this directory structure: app/ __init__.py sub1/ __init__.py mod1.py sub2/ __init__.py mod2.py I'm coding mod1, and I need to import something from mod2. How should I do it? I tried from ..sub2 import mod2 but I'm getting an "Attempted relative import in non-package". I googled around but found only "sys.path manipulation" hacks. Isn't there a clean way? Edit: all my __init__.py's are currently empty Edit2: I'm trying to do this because sub2 contains classes that are shared across sub packages (sub1, subX, etc.). Edit3: The behaviour I'm looking for is the same as described in PEP 366 (thanks John B)

    Read the article

  • Faster float to int conversion in Python

    - by culebrón
    Here's a piece of code that takes most time in my program, according to timeit statistics. It's a dirty function to convert floats in [-1.0, 1.0] interval into unsigned integer [0, 2**32]. How can I accelerate floatToInt? piece = [] rng = range(32) for i in rng: piece.append(1.0/2**i) def floatToInt(x): n = x + 1.0 res = 0 for i in rng: if n >= piece[i]: res += 2**(31-i) n -= piece[i] return res

    Read the article

  • Python : Small Regex problem

    - by user316758
    Hi, when I try to extract this video ID (AIiMa2Fe-ZQ) with a regex expression, I can't get the dash an all the letters after. Someone can help me please? Thanks >>> id = re.search('(?<=\?v\=)\w+', 'http://www.youtube.com/watch?v=AIiMa2Fe-ZQ') >>> print id.group(0) >>> AIiMa2Fe

    Read the article

  • Getting error on inserting tuple values in postgreSQL table using python

    - by rahman.bd
    Hello, I want to keep last.fm's user recent music tracks list to postgresql database table using pylast interface.But when I tried to insert values to the table it shows errors.Code example: for i, artist in enumerate(recent_tracks): for key in sorted(artist): cur.execute(""" INSERT INTO u_recent_track(Playback_date,Time_stamp,Track) VALUES (%s,%s)""", (key, artist[key])) conn.commit() cur.execute("SELECT * FROM u_recent_track;") cur.fetchone() for row in cur: print ' '.join(row[1:]) cur.close() conn.close() Here "recent_tracks" tuple have the values for example: artist 0 - playback_date : 5 May 2010, 11:14 - timestamp : 1273058099 - track : Brian Eno - Web I want to store these value under u_recent_track(Tid,Playback_date,Time_stamp,Track).Can anybody have idea how to sort out this problem? when I tried to run, it shows error: Traceback (most recent call last): File "F:\JavaWorkspace\Test\src\recent_track_database.py", line 50, in <module> VALUES (%s,%s,%s)""", (key, artist[key])) IndexError: tuple index out of range Thanks in advanced!

    Read the article

  • Python (Django). Store telnet connection

    - by Shamanu4
    Hello. I am programming web interface which communicates with cisco switches via telnet. I want to make such system which will be storing one telnet connection per switch and every script (web interface, cron jobs, etc.) will have access to it. This is needed to make a single query queue for each device and prevent huge cisco processor load caused by several concurent telnet connections. How do I can do this?

    Read the article

  • python lxml problem

    - by David ???
    I'm trying to print/save a certain element's HTML from a web-page. I've retrieved the requested element's XPath from firebug. All I wish is to save this element to a file. I don't seem to succeed in doing so. (tried the XPath with and without a /text() at the end) I would appreciate any help, or past experience. 10x, David import urllib2,StringIO from lxml import etree url='http://www.tutiempo.net/en/Climate/Londres_Heathrow_Airport/12-2009/37720.htm' seite = urllib2.urlopen(url) html = seite.read() seite.close() parser = etree.HTMLParser() tree = etree.parse(StringIO.StringIO(html), parser) xpath = "/html/body/table/tbody/tr/td[2]/div/table/tbody/tr[6]/td/table/tbody/tr/td[3]/table/tbody/tr[3]/td/table/tbody/tr/td/table/tbody/tr/td/table/tbody/text()" elem = tree.xpath(xpath) print elem[0].strip().encode("utf-8")

    Read the article

  • Python: Script works, but seems to deadlock after some time

    - by sberry2A
    I have the following script, which is working for the most part Link to PasteBin The script's job is to start a number of threads which in turn each start a subprocess with Popen. The output from each subprocess is as follows: 1 2 3 . . . n Done Bascially the subprocess is transferring 10M records from tables in one database to different tables in another db with a lot of data massaging/manipulation in between because of the different schemas. If the subprocess fails at any time in it's execution (bad records, duplicate primary keys, etc), or it completes successfully, it will output "Done\n". If there are no more records to select against for transfer then it will output "NO DATA\n" My intent was to create my script "tableTransfer.py" which would spawn a number of these processes, read their output, and in turn output information such as number of updates completed, time remaining, time elapsed, and number of transfers per second. I started running the process last night and checked in this morning to see it had deadlocked. There were not subprocceses running, there are still records to be updated, and the script had not exited. It was simply sitting there, no longer outputting the current information because no subprocces were running to update the total number complete which is what controls updates to the output. This is running on OS X. I am looking for three things: I would like to get rid of the possibility of this deadlock occurring so I don't need to check in on it as frequently. Is there some issue with locking? Am I doing this in a bad way (gThreading variable to control looping of spawning additional thread... etc.) I would appreciate some suggestions for improving my overall methodology. How should I handle ctrl-c exit? Right now I need to kill the process, but assume I should be able to use the signal module or other to catch the signal and kill the threads, is that right? I am not sure whether I should be pasting my entire script here, since I usually just paste snippets. Let me know if I should paste it here as well.

    Read the article

  • FIFO dequeueing in python?

    - by Aaron Ramsey
    hello again everybody— I'm looking to make a functional (not necessarily optimally efficient, as I'm very new to programming) FIFO queue, and am having trouble with my dequeueing. My code looks like this: class QueueNode: def __init__(self, data): self.data = data self.next = None def __str__(self): return str(self.data) class Queue: def__init__(self): self.front = None self.rear = None self.size = 0 def enqueue(self, item) newnode = QueueNode(item) newnode.next = None if self.size == 0: self.front = self.rear = newnode else: self.rear = newnode self.rear.next = newnode.next self.size = self.size+1 def dequeue(self) dequeued = self.front.data del self.front self.size = self.size-1 if self.size == 0: self.rear = None print self.front #for testing if I do this, and dequeue an item, I get the error "AttributeError: Queue instance has no attribute 'front'." I guess my function doesn't properly assign the new front of the queue? I'm not sure how to fix it though. I don't really want to start from scratch, so if there's a tweak to my code that would work, I'd prefer that—I'm not trying to minimize runtime so much as just get a feel for classes and things of that nature. Thanks in advance for any help, you guys are the best.

    Read the article

  • Read a file on App Eninge with Python?

    - by PanosJee
    Is it possible to open a file on GAE just to read its contents and get the last modified tag? I get a IOError: [Errno 13] file not accessible: I know that i cannot delete or update but i believe reading should be possible Has anyone faced a similar problem? os.stat(f,'r').st_mtim

    Read the article

  • Python Decorators and inheritance

    - by wheaties
    Help a guy out. Can't seem to get a decorator to work with inheritance. Broke it down to the simplest little example in my scratch workspace. Still can't seem to get it working. class bar(object): def __init__(self): self.val = 4 def setVal(self,x): self.val = x def decor(self, func): def increment(self, x): return func( self, x ) + self.val return increment class foo(bar): def __init__(self): bar.__init__(self) @decor def add(self, x): return x Oops, name "decor" is not defined. Okay, how about @bar.decor? TypeError: unbound method "decor" must be called with a bar instance as first argument (got function instance instead) Ok, how about @self.decor? Name "self" is not defined. Ok, how about @foo.decor?! Name "foo" is not defined. AaaaAAaAaaaarrrrgggg... What am I doing wrong?

    Read the article

  • Python/PyParsing: Difficulty with setResultsName

    - by Rosarch
    I think I'm making a mistake in how I call setResultsName(): from pyparsing import * DEPT_CODE = Regex(r'[A-Z]{2,}').setResultsName("Dept Code") COURSE_NUMBER = Regex(r'[0-9]{4}').setResultsName("Course Number") COURSE_NUMBER.setParseAction(lambda s, l, toks : int(toks[0])) course = DEPT_CODE + COURSE_NUMBER course.setResultsName("course") statement = course From IDLE: >>> myparser import * >>> statement.parseString("CS 2110") (['CS', 2110], {'Dept Code': [('CS', 0)], 'Course Number': [(2110, 1)]}) The output I hope for: >>> myparser import * >>> statement.parseString("CS 2110") (['CS', 2110], {'Course': ['CS', 2110], 'Dept Code': [('CS', 0)], 'Course Number': [(2110, 1)]}) Does setResultsName() only work for terminals?

    Read the article

  • python: send a list/dict over network

    - by facha
    Hi, everyone I'm looking for an easy way of packing/unpacking data structures for sending over the network: on client just before sending: a = ((1,2),(11,22,),(111,222)) message = pack(a) and then on server: a = unpack(message) Is there a library that could do pack/unpack magic? Thanks in advance

    Read the article

< Previous Page | 151 152 153 154 155 156 157 158 159 160 161 162  | Next Page >