Search Results

Search found 13602 results on 545 pages for 'python decorators'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 367/545 | < Previous Page | 363 364 365 366 367 368 369 370 371 372 373 374  | Next Page >

  • Scrapy Not Returning Additonal Info from Scraped Link in Item via Request Callback

    - by zoonosis
    Basically the code below scrapes the first 5 items of a table. One of the fields is another href and clicking on that href provides more info which I want to collect and add to the original item. So parse is supposed to pass the semi populated item to parse_next_page which then scrapes the next bit and should return the completed item back to parse Running the code below only returns the info collected in parse If I change the return items to return request I get a completed item with all 3 "things" but I only get 1 of the rows, not all 5. Im sure its something simple, I just can't see it. class ThingSpider(BaseSpider): name = "thing" allowed_domains = ["somepage.com"] start_urls = [ "http://www.somepage.com" ] def parse(self, response): hxs = HtmlXPathSelector(response) items = [] for x in range (1,6): item = ScrapyItem() str_selector = '//tr[@name="row{0}"]'.format(x) item['thing1'] = hxs.select(str_selector")]/a/text()').extract() item['thing2'] = hxs.select(str_selector")]/a/@href').extract() print 'hello' request = Request("www.nextpage.com", callback=self.parse_next_page,meta={'item':item}) print 'hello2' request.meta['item'] = item items.append(item) return items def parse_next_page(self, response): print 'stuff' hxs = HtmlXPathSelector(response) item = response.meta['item'] item['thing3'] = hxs.select('//div/ul/li[1]/span[2]/text()').extract() return item

    Read the article

  • need help in site classification

    - by goh
    hi guys, I have to crawl the contents of several blogs. The problem is that I need to classify whether the blogs the authors are from a specific school and is talking about the school's stuff. May i know what's the best approach in doing the crawling or how should i go about the classification?

    Read the article

  • Is django orm & templates thread safe?

    - by Piotr Czapla
    I'm using django orm and templates to create a background service that is ran as management command. Do you know if django is thread safe? I'd like to use threads to speed up processing. The processing is blocked by I/O not CPU so I don't care about performance hit caused by GIL.

    Read the article

  • Condition checking vs. Exception handling

    - by Aidas Bendoraitis
    When is exception handling more preferable than condition checking? There are many situations where I can choose using one or the other. For example, this is a summing function which uses a custom exception: # module mylibrary class WrongSummand(Exception): pass def sum_(a, b): """ returns the sum of two summands of the same type """ if type(a) != type(b): raise WrongSummand("given arguments are not of the same type") return a + b # module application using mylibrary from mylibrary import sum_, WrongSummand try: print sum_("A", 5) except WrongSummand: print "wrong arguments" And this is the same function, which avoids using exceptions # module mylibrary def sum_(a, b): """ returns the sum of two summands if they are both of the same type """ if type(a) == type(b): return a + b # module application using mylibrary from mylibrary import sum_ c = sum_("A", 5) if c is not None: print c else: print "wrong arguments" I think that using conditions is always more readable and manageable. Or am I wrong? What are the proper cases for defining APIs which raise exceptions and why?

    Read the article

  • How to build a Django form which requires a delay to be re-submitted ?

    - by pierre-guillaume-degans
    Hey, In order to avoid spamming, I would like to add a waiting time to re-submit a form (i.e. the user should wait a few seconds to submit the form, except the first time that this form is submitted). To do that, I added a timestamp to my form (and a security_hash field containing the timestamp plus the settings.SECRET_KEY which ensures that the timestamp is not fiddled with). This look like: class MyForm(forms.Form): timestamp = forms.IntegerField(widget=forms.HiddenInput) security_hash = forms.CharField(min_length=40, max_length=40, widget=forms.HiddenInput) # + some other fields.. # + methods to build the hash and to clean the timestamp... # (it is based on django.contrib.comments.forms.CommentSecurityForm) def clean_timestamp(self): """Make sure the delay is over (5 seconds).""" ts = self.cleaned_data["timestamp"] if not time.time() - ts > 5: raise forms.ValidationError("Timestamp check failed") return ts # etc... This works fine. However there is still an issue: the timestamp is checked the first time the form is submitted by the user, and I need to avoid this. Any idea to fix it ? Thank you ! :-)

    Read the article

  • basic unique ModelForm field for Google App Engine

    - by Alexander Vasiljev
    I do not care about concurrency issues. It is relatively easy to build unique form field: from django import forms class UniqueUserEmailField(forms.CharField): def clean(self, value): self.check_uniqueness(super(UniqueUserEmailField, self).clean(value)) def check_uniqueness(self, value): same_user = users.User.all().filter('email', value).get() if same_user: raise forms.ValidationError('%s already_registered' % value) so one could add users on-the-fly. Editing existing user is tricky. This field would not allow to save user having other user email. At the same time it would not allow to save a user with the same email. What code do you use to put a field with uniqueness check into ModelForm?

    Read the article

  • Dynamically setting the queryset of a ModelMultipleChoiceField to a custom recordset

    - by Daniel Quinn
    I've seen all the howtos about how you can set a ModelMultipleChoiceField to use a custom queryset and I've tried them and they work. However, they all use the same paradigm: the queryset is just a filtered list of the same objects. In my case, I'm trying to get the admin to draw a multiselect form that instead of using usernames as the text portion of the , I'd like to use the name field from my account class. Here's a breakdown of what I've got: # models.py class Account(models.Model): name = models.CharField(max_length=128,help_text="A display name that people understand") user = models.ForeignKey(User, unique=True) # Tied to the User class in settings.py class Organisation(models.Model): administrators = models.ManyToManyField(User) # admin.py from django.forms import ModelMultipleChoiceField from django.contrib.auth.models import User class OrganisationAdminForm(forms.ModelForm): def __init__(self, *args, **kwargs): from ethico.accounts.models import Account self.base_fields["administrators"] = ModelMultipleChoiceField( queryset=User.objects.all(), required=False ) super(OrganisationAdminForm, self).__init__(*args, **kwargs) class Meta: model = Organisation This works, however, I want queryset above to draw a selectbox with the Account.name property and the User.id property. This didn't work: queryset=Account.objects.all().order_by("name").values_list("user","name") It failed with this error: 'tuple' object has no attribute 'pk' I figured that this would be easy, but it's turned into hours of dead-ends. Anyone care to shed some light?

    Read the article

  • Returning Database Blobs in TurboGears 2.x / FCGI / Lighttpd extremely slow

    - by Tom
    Hey everyone, I am running a TG2 App on lighttpd via flup/fastcgi. We are reading images (~30kb each) from BlobFields in a MySQL database and return those images with a custom mime type via a controller method. Caching these images on the hard disk makes no sense because they change with every request, the only reason we cache these in the DB is that creating these images is quite expensive and the data used to create the images is also present in plain text on the website. Now to the problem itself: When returning such an image, things get extremely slow. The code runs totally fine on paster itself with no visible delay, but as soon as its running via fcgi/lighttpd the described phenomenon happens. I profiled the method of my controller that returns my blob, and the entire method runs in a few miliseconds, but when "return" executes, the entire app hangs for roughly 10 seconds. We could not reproduce the same error with PHP on FCGI. This only seems to happen with Turbogears or Pylons. Here for your consideration the concerned piece of source code: @expose(content_type=CUSTOM_CONTENT_TYPE) def return_img(self, img_id): """ Return a DB persisted image when requested """ img = model.Images.by_id(img_id) #get image from DB response.headers['content-type'] = 'image/png' return img.data # this causes the app to hang for 10 seconds

    Read the article

  • PyParsing: Not all tokens passed to setParseAction()

    - by Rosarch
    I'm parsing sentences like "CS 2110 or INFO 3300". I would like to output a format like: [[("CS" 2110)], [("INFO", 3300)]] To do this, I thought I could use setParseAction(). However, the print statements in statementParse() suggest that only the last tokens are actually passed: >>> statement.parseString("CS 2110 or INFO 3300") Match [{Suppress:("or") Re:('[A-Z]{2,}') Re:('[0-9]{4}')}] at loc 7(1,8) string CS 2110 or INFO 3300 loc: 7 tokens: ['INFO', 3300] Matched [{Suppress:("or") Re:('[A-Z]{2,}') Re:('[0-9]{4}')}] -> ['INFO', 3300] (['CS', 2110, 'INFO', 3300], {'Course': [(2110, 1), (3300, 3)], 'DeptCode': [('CS', 0), ('INFO', 2)]}) I expected all the tokens to be passed, but it's only ['INFO', 3300]. Am I doing something wrong? Or is there another way that I can produce the desired output? Here is the pyparsing code: from pyparsing import * def statementParse(str, location, tokens): print "string %s" % str print "loc: %s " % location print "tokens: %s" % tokens DEPT_CODE = Regex(r'[A-Z]{2,}').setResultsName("DeptCode") COURSE_NUMBER = Regex(r'[0-9]{4}').setResultsName("CourseNumber") OR_CONJ = Suppress("or") COURSE_NUMBER.setParseAction(lambda s, l, toks : int(toks[0])) course = DEPT_CODE + COURSE_NUMBER.setResultsName("Course") statement = course + Optional(OR_CONJ + course).setParseAction(statementParse).setDebug()

    Read the article

  • How can I detect whether an image is a PNG or APNG format?

    - by perlit
    APNG is backwards compatible with PNG. I opened up an apng and png file in a hex editor and the first few bytes look identical. So if a user uploads either of these formats, how do I detect what the format really is? I've seen this done on some sites that block apng. I'm guessing the ImageMagick library makes this easy, but what if I were to do the detect without the use of an image processing library (for learning purposes)? Can I look for specific bytes that tell me if the file is apng? Solutions in any language is welcome.

    Read the article

  • Why wont numpy matrix let me print its rows?

    - by uberjumper
    Okay this is probably a really dumb question, however its really starting to hurt. I have a numpy matrix, and basically i print it out row by row. However i want to make each row be formatted and separated properly. >>> arr = numpy.matrix([[x for x in range(5)] for y in range(5)]) >>> arr matrix([[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]) Lets say i want to print the first row, and add a '|' between each element: >>> '|'.join(map(str, arr[0,])) '[[0 1 2 3 4]]' Err... >>> '|'.join(map(lambda x: str(x[0]), arr[0])) '[[0 1 2 3 4]]' I am really confused by this behavior why does it do this?

    Read the article

  • twisted reactor stops too early

    - by pygabriel
    I'm doing a batch script to connect to a tcp server and then exiting. My problem is that I can't stop the reactor, for example: cmd = raw_input("Command: ") # custom factory, the protocol just send a line reactor.connectTCP(HOST,PORT, CommandClientFactory(cmd) d = defer.Deferred() d.addCallback(lambda x: reactor.stop()) reactor.callWhenRunning(d.callback,None) reactor.run() In this code the reactor stops before that the tcp connection is done and the cmd is passed. How can I stop the reactor after that all the operation are finished?

    Read the article

  • Sqlalchemy complex in_ clause

    - by lostlogic
    I'm trying to find a way to cause sqlalchemy to generate sql of the following form: select * from t where (a,b) in ((a1,b1),(a2,b2)); Is this possible? If not, any suggestions on a way to emulate it? Thanks kindly!

    Read the article

  • django: Changing auto_id of ModelForm based form class

    - by Meilo
    Every time I create an instance of the TestForm specified below, I have to overwrite the standard id format with auto_id=True. How can this be done once only in the form class instead? Any hints are very welcome. views.py from django.forms import ModelForm from models import Test class TestForm(ModelForm): class Meta: model = Test def test(request): form = TestForm(auto_id=True)

    Read the article

  • Auto filling polymorphic table on save or on delete in django

    - by Mo J. Mughrabi
    Hi, Am working on an project in which I made an app "core" it will contain some of the reused models across my projects, most of those are polymorphic models (Generic content types) and will be linked to different models. Example below am trying to create audit model and will be linked to several models which may require auditing. This is the polls/models.py from django.db import models from django.contrib.auth.models import User from core.models import * from django.contrib.contenttypes import generic class Poll(models.Model): ## TODO: Document question = models.CharField(max_length=300) question_slug=models.SlugField(editable=False) start_poll_at = models.DateTimeField(null=True) end_poll_at = models.DateTimeField(null=True) is_active = models.BooleanField(default=True) audit_obj=generic.GenericRelation(Audit) def __unicode__(self): return self.question class Choice(models.Model): ## TODO: Document choice = models.CharField(max_length=200) poll=models.ForeignKey(Poll) audit_obj=generic.GenericRelation(Audit) class Vote(models.Model): ## TODO: Document choice=models.ForeignKey(Choice) Ip_Address=models.IPAddressField(editable=False) vote_at=models.DateTimeField("Vote at", editable=False) here is the core/modes.py from django.db import models from django.contrib.auth.models import User from django.contrib.contenttypes.models import ContentType from django.contrib.contenttypes import generic class Audit(models.Model): ## TODO: Document # Polymorphic model using generic relation through DJANGO content type created_at = models.DateTimeField("Created at", auto_now_add=True) created_by = models.ForeignKey(User, db_column="created_by", related_name="%(app_label)s_%(class)s_y+") updated_at = models.DateTimeField("Updated at", auto_now=True) updated_by = models.ForeignKey(User, db_column="updated_by", null=True, blank=True, related_name="%(app_label)s_%(class)s_y+") content_type = models.ForeignKey(ContentType) object_id = models.PositiveIntegerField(unique=True) content_object = generic.GenericForeignKey('content_type', 'object_id') and here is polls/admin.py from django.core.context_processors import request from polls.models import Poll, Choice from core.models import * from django.contrib import admin class ChoiceInline(admin.StackedInline): model = Choice extra = 3 class PollAdmin(admin.ModelAdmin): inlines = [ChoiceInline] admin.site.register(Poll, PollAdmin) Am quite new to django, what am trying to do here, insert a record in audit when a record is inserted in polls and then update that same record when a record is updated in polls.

    Read the article

  • Help calling class from a class above.

    - by wtzolt
    Hello, How to call from class oneThread: back to class fun:? As in, address a class written below. Is it possible? class oneThread(threading.Thread): def __init__(self): threading.Thread.__init__(self) self.start() def run(self): print "1" time.sleep(1) print "2" time.sleep(1) print "3" self.wTree.get_widget("entryResult").set_text("Done with One.") # How to call from here back to class fun, which of course is below...? class fun: wTree = None def __init__( self ): self.wTree = gtk.glade.XML( "main.glade" ) self.wTree.signal_autoconnect( {"on_buttonOne" : self.one} ) gtk.main() def one(self, widget): oneThread(); gtk.gdk.threads_init() do=fun()

    Read the article

  • has any tools easy to download or uploaed data from gae ..

    - by zjm1126
    i find this: http://aralbalkan.com/1784 but it is : Gaebar is an easy-to-use, standalone Django application that you can plug in to your existing Google App Engine Django or app-engine-patch-based Django applications on Google App Engine to give them datastore backup and restore functionality. my app is not based on django,so did you know any tools esay to do this . thanks

    Read the article

  • Getting unpredictable data into a tabular format

    - by Acorn
    The situation: Each page I scrape has <input> elements with a title= and a value= I don't know what is going to be on the page. I want to have all my collected data in a single table at the end, with a column for each title. So basically, I need each row of data to line up with all the others, and if a row doesn't have a certain element, then it should be blank (but there must be something there to keep the alignment). eg. First page has: {animal: cat, colour: blue, fruit: lemon, day: monday} Second page has: {animal: fish, colour: green, day: saturday} Third page has: {animal: dog, number: 10, colour: yellow, fruit: mango, day: tuesday} Then my resulting table should be: animal | number | colour | fruit | day cat | none | blue | lemon | monday fish | none | green | none | saturday dog | 10 | yellow | mango | tuesday Although it would be good to keep the order of the title value pairs, which I know dictionaries wont do. So basically, I need to generate columns from all the titles (kept in order but somehow merged together) What would be the best way of going about this without knowing all the possible titles and explicitly specifying an order for the values to be put in?

    Read the article

  • How to make scipy.interpolate give a an extrapolated result beyond the input range?

    - by Salim Fadhley
    I'm trying to port a program which uses a hand-rolled interpolator (developed by a mathematitian colleage) over to use the interpolators provided by scipy. I'd like to use or wrap the scipy interpolator so that it has as close as possible behavior to the old interpolator. A key difference between the two functions is that in our original interpolator - if the input value is above or below the input range, our original interpolator will extrapolate the result. If you try this with the scipy interpolator it raises a ValueError. Consider this program as an example: import numpy as np from scipy import interpolate x = np.arange(0,10) y = np.exp(-x/3.0) f = interpolate.interp1d(x, y) print f(9) print f(11) # Causes ValueError, because it's greater than max(x) Is there a sensible way to make it so that instead of crashing, the final line will simply do a linear extrapolate, continuing the gradients defined by the first and last two pouints to infinity. Note, that in the real software I'm not actually using the exp function - that's here for illustration only!

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 363 364 365 366 367 368 369 370 371 372 373 374  | Next Page >