the following function parses a CSV file into a list of dictionaries, where each element in the list is a dictionary where the values are indexed by the header of the file (assumed to be the first line.)
this function is very very slow, taking ~6 seconds for a file that's relatively small (less than 30,000 lines.)
how can I speed it up?
def csv2dictlist_raw(filename, delimiter='\t'):
f = open(filename)
header_line = f.readline().strip()
header_fields = header_line.split(delimiter)
dictlist = []
# convert data to list of dictionaries
for line in f:
values = map(tryEval, line.strip().split(delimiter))
dictline = dict(zip(header_fields, values))
dictlist.append(dictline)
return (dictlist, header_fields)
thanks.
In a class method, I can add attributes using the built-in function:
setattr(self, "var_name", value).
If I want to do the same thing within a module, I can do something like:
globals()["var_name"] = value
Is this the best way to do this, or is there a more pythonic solution?
"8,5,,1,4,7,,,,7,,1,9,3,6,,,8,6,3,9,,2,5,4,,,,,3,2,,,7,4,1,1,,4,,6,9,,5,,,,5,,,1,,6,3,,,6,5,,,,7,4,,1,7,6,,,,8,,5,,,7,1,,3,9,"
I'm doing a programming challenge where i need to parse this sequence into my sudoku script.
Need to get the above sequence into 8,5,0,1,4,7,0,0,0,7,0,1,9,3,6,0,0,8.........
I tried re but without success, help is appreciated, thanks.
Hello all ,
I have a string like "SAB_bARGS_D" . What I want is that the string gets divided into list of characters but whenever there is a _ sign the next character gets appended to the previous one.
So the answer to above should be ['S','A','B_b','A','R','G','S_D']
It can be done by using a for loop traversing through the list but is there an inbuilt function that I can use.....
Thanks a lot
This is probably a really silly question but, given the example code at the bottom, how would I get a single list that retain the tuples?
(I've looked at the itertools but it flattens everything)
What I currently get is:
('id', 20, 'integer')
('companyname', 50, 'text')
[('focus', 30, 'text'), ('fiesta',
30, 'text'), ('mondeo', 30, 'text'),
('puma', 30, 'text')]
('contact', 50, 'text')
('email', 50, 'text')
what I would like is a single level list like:
('id', 20, 'integer')
('companyname', 50, 'text')
('focus', 30, 'text')
('fiesta', 30, 'text')
('mondeo', 30, 'text')
('puma', 30, 'text')
('contact', 50, 'text')
('email', 50, 'text')
def getproducts():
temp_list=[]
product_list=['focus','fiesta','mondeo','puma'] #usually this would come from a db
for p in product_list:
temp_list.append((p,30,'text'))
return temp_list
def createlist():
column_title_list = (
("id",20,"integer"),
("companyname",50,"text"),
getproducts(),
("contact",50,"text"),
("email",50,"text"),
)
return column_title_list
for item in createlist():
print item
Thanks
ALJ
I have a model called "Activity" in my django app. in the admin interface, it appears on the screen as "Activitys". how can I override the label on the admin page to make it "Activities" instead?
I see in the archives how to do this for a field, but not for a model itself. thanks!
I am reading one line at a time from a file, but at the end of each line it adds a '\n'.
example:
line is: 094 234 hii
but my input is: 094 234 hii\n
I want to read line by linem but I don't need to keep the newlines...
My goal is to read a list from every line: I need ['094','234','hii'], not ['094','234','hii\n']
Any advice?
I've increased the font of my ticklabels successfully, but now they're too close to the axis. I'd like to add a little breathing room between the ticklabels and the axis.
sorry, im sure this is asked a bunch, but i couldnt find it.
in myModule.py:
from myModule.subModule import myClass
i am working on myClass, and want to stay in my ipython session and test it. reload(myModule) doesnt re-compile myClass.
how can i do this?
What I wanted is printing out 5 dots that a dot printed per a second using time.sleep(), but the result was 5 dots were printed at once after 5 seconds delay.
Tried both print and sys.stdout.write, same result.
Thanks for any advices.
import time
import sys
def wait_for(n):
"""Wait for {n} seconds. {n} should be an integer greater than 0."""
if not isinstance(n, int):
print 'n in wait_for(n) should be an integer.'
return
elif n < 1:
print 'n in wait_for(n) should be greater than 0.'
return
for i in range(0, n):
sys.stdout.write('.')
time.sleep(1)
sys.stdout.write('\n')
def main():
wait_for(5) # FIXME: doesn't work as expected
if __name__ == '__main__':
try:
main()
except KeyboardInterrupt:
print '\nAborted.'
"insert into NodeProfileSections (profile_no, tpl_section_no)
select (np.profile_no, tps.tpl_section_no)
from NodeProfile np, TemplateProfileSection tps, TemplateProfile tp
where np.hostname = '%s' AND np.role = '%s' AND tp.tpl_profile_no = tps.tpl_profile_no
AND tp.tpl_name = '%s' AND tp.role = '%s' AND tps.tpl_section_name = '%s';"
%(hostname, role, template_name, role, section_name)
error_message = 'Operand should contain 1 column(s)'
How to solve this problem?
Starting from an Html input like this:
<p>
<a href="http://www.foo.com">this if foo</a>
<a href="http://www.bar.com">this if bar</a>
</p>
using BeautifulSoup, i would like to change this Html in:
<p>
<a href="http://www.foo.com">this if foo</a><b>OK</b>
<a href="http://www.bar.com">this if bar</a><b>OK</b>
</p>
Is it possible to do this using BeautifulSoup?
I need to control a program by sending commands in utf-8 encoding to its standard input. For this I run the program using subprocess.Popen():
proc = Popen("myexecutable.exe", shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE)
proc.stdin.write(u'ééé'.encode('utf_8'))
If I run this from a cygwin utf-8 console, it works. If I run it from a windows console (encoding ='cp1252') this doesn't work. Is there a way to make this work without having to install a cygwin utf-8 console on each computer I want it to run from ? (NB: I don't need to output anything to console)
I am trying to use the ModelForm to add my data. It is working well, except that the ForeignKey dropdown list is showing all values and I only want it to display the values that a pertinent for the logged in user.
Here is my model for ExcludedDate, the record I want to add:
class ExcludedDate(models.Model):
date = models.DateTimeField()
reason = models.CharField(max_length=50)
user = models.ForeignKey(User)
category = models.ForeignKey(Category)
recurring = models.ForeignKey(RecurringExclusion)
def __unicode__(self):
return self.reason
Here is the model for the category, which is the table containing the relationship that I'd like to limit by user:
class Category(models.Model):
name = models.CharField(max_length=50)
user = models.ForeignKey(User, unique=False)
def __unicode__(self):
return self.name
And finally, the form code:
class ExcludedDateForm(ModelForm):
class Meta:
model = models.ExcludedDate
exclude = ('user', 'recurring',)
How do I get the form to display only the subset of categories where category.user equals the logged in user?
I have the following code in one of my models
class PostImage(models.Model):
post = models.ForeignKey(Post, related_name="images")
# @@@@ figure out a way to have image folders per user...
image = models.ImageField(upload_to='images')
image_infowindow = models.ImageField(upload_to='images')
image_thumb = models.ImageField(upload_to='images')
image_web = models.ImageField(upload_to='images')
description = models.CharField(max_length=100)
order = models.IntegerField(null=True)
IMAGE_SIZES = {
'image_infowindow':(70,70),
'image_thumb':(100,100),
'image_web':(640,480),
}
def delete(self, *args, **kwargs):
# delete files..
self.image.delete(save=False)
self.image_thumb.delete(save=False)
self.image_web.delete(save=False)
self.image_infowindow.delete(save=False)
super(PostImage, self).delete(*args, **kwargs)
I am trying to delete the files when the delete() method is called on PostImage. However, the files are not being removed.
As you can see, I am overriding the delete() method, and deleting each ImageField. For some reason however, the files are not being removed.
this is gaema : http://code.google.com/p/gaema/
this demo : http://code.google.com/p/gaema/source/browse/#hg/demos/webapp
only has Google Accounts
i want demo for Facebook and Twitter
thanks
How can I use the wx.ProgressDialog to time my method called imgSearch? The imgSearch method finds image files on the user's pc. How can I make the wx.ProgressDialog run while imgSearch is still running and display how long the imgSearch is taking?
Here's my code:
def onFind (self,event)# triggered by a button click
max = 80
dlg = wx.ProgressDialog("Progress dialog example","An informative message",parent=self, style = wx.PD_CAN_ABORT| wx.PD_APP_MODAL| wx.PD_ELAPSED_TIME| wx.PD_REMAINING_TIME)
keepGoing = True
count = 0
imageExtentions = ['*.jpg', '*.jpeg', '*.png', '*.tif', '*.tiff']
selectedDir = 'C:\\'
imgSearch.findImages(imageExtentions, selectedDir)# my method
while keepGoing and count < max:
count += 1
wx.MilliSleep(250)
if count >= max / 2:
(keepGoing, skip) = dlg.Update(count, "Half-time!")
else:
(keepGoing, skip) = dlg.Update(count)
dlg.Destroy()
hey, I am starting a project and wonder the relationship between the characters in images and the whole web page where the images reside. so first, i want to crawl some images and their web pages.....need to save the crawl result in local disk for further analysis.
I wonder if there is any open source for this issue?
thx^_^
Hello.
I have 2 different tables in my database. They have some variables common and some different. For example:
Table1:
ID
Date
Name
Address
Fax
Table2:
ID
Date
Name
e-mail
Telephone number
I want to display data together sorted by date & ID but from both tables. For example, first displayed will be the newest record from first table, but the second one will be the record from another table posted right after first one.
Hope everybody understand, sorry for my English.
Cheers.
I'm using a gridbagsizer to place two panels within a frame.
control_panel = wx.Panel(self, id=ID_CONTROL_PANEL)
main_panel = wx.Panel(self, id=ID_MAIN_PANEL)
frame_sizer = wx.GridBagSizer(1, 1)
frame_sizer.Add(control_panel, (0, 0), (2, 5), flag=wx.EXPAND)
frame_sizer.Add(main_panel, (2, 0), (5, 5), flag=wx.EXPAND)
frame_sizer.AddGrowableRow(0)
frame_sizer.AddGrowableCol(0)
self.SetSizer(frame_sizer)
I want the control_panel to be above the main_panel (as you can see above). However this is the output I get:
I don't know why the top panel is so much larger than the bottom panel, when I've specified the opposite. (2, 5) for the top and (5, 5) for the bottom. It also has a strange behaviour when I resize it smaller, it basically gives me what I want (see image below), but I don't know how to maintain the ratio when I make it larger again.
However when you start resizing larger (even by a small amount) you can see (below) the shapes and ratio change dramatically in the other direction (with the bottom becoming much smaller and the top much larger).
Still trying to learn this sizers and can't really find this problem with any of the examples I've seen. Also tried experimenting with all the parameters and doing some tests and I can't figure out what's going on here. Any help would be much appreciated.
I have a set of points like: pointA(3302.34,9392.32), pointB(34322.32,11102.03), etc.
I need to scale these so each x- and y-coordinate is in the range (0.0 - 1.0).
I tried doing this by first finding the largest x value in the data set (maximum_x_value), and the largest y value in the set (minimum_y_value). I then did the following:
pointA.x = (pointA.x - minimum_x_value) / (maximum_x_value - minimum_x_value)
pointA.y = (pointA.y - minimum_y_value) / (maximum_y_value - minimum_y_value)
This changes the relative distances(?), and therefore makes the data useless for my purposes. Is there a way to scale these coordinates while keeping their relative distances the intact?
I have a string "one two 9three 52eight four", so I only want to get "one two four", because "three" starts with "9" and "eight" starts with "52".
I tried:
"(?!\d)\w+"
but it's still taking the "three" and "eight". I don't want it.