Search Results

Search found 27118 results on 1085 pages for 'mysql python'.

Page 150/1085 | < Previous Page | 146 147 148 149 150 151 152 153 154 155 156 157  | Next Page >

• Learning Python and trying to get first two letters and last two letters of a string.

  - by Sergio Tapia

  Here's my code: # B. both_ends # Given a string s, return a string made of the first 2 # and the last 2 chars of the original string, # so 'spring' yields 'spng'. However, if the string length # is less than 2, return instead the empty string. def both_ends(s): if len(s) <= 2: return "" else: return s[0] + s[1] + s[len(s)-2] + s[len(s-1)] # +++your code here+++ return Unfortunately my program doesn't run. :( I'm sure I'm overlooking something since I'm a newbie with Python. Here's the error: > Traceback (most recent call last): File "C:\Users\Sergio\Desktop\google-python-exercises\google-python-exercises\basic\string1.py", line 120, in <module> main() File "C:\Users\Sergio\Desktop\google-python-exercises\google-python-exercises\basic\string1.py", line 97, in main test(both_ends('spring'), 'spng') File "C:\Users\Sergio\Desktop\google-python-exercises\google-python-exercises\basic\string1.py", line 44, in both_ends return s[0] + s[1] + s[len(s)-2] + s[len(s-1)] TypeError: unsupported operand type(s) for -: 'str' and 'int' Thanks for the help guys. :D

  Read the article

• How to download the Enthought Tool Suite python script?

  - by user1539217

  I have Python downloaded already, but want to contribute code to the Enthought Tool Suite (ETS). According to this site (http://code.enthought.com/source/): it says to download the following python script: https://raw.github.com/enthought/ets/master/ets.py I put the ets.py into the python script, save as...and in the Python Shell, I typed in "import ets" However, nothing happens. Also, the site says to run the following commands: $ mkdir ets $ cd ets # and copy ets.py here $ python ets.py clone Typing those lines in Python Shell gives me the message: "SyntaxError: invalid syntax" As you can see, I'm new to Python, and I don't know what I'm doing. How do I download the ETS script and run the commands??

  Read the article

• How can I get around MySQL Errcode 13 with SELECT INTO OUTFILE?

  - by Ryan Olson

  but I am trying to dump the contents of a table to a csv file using a MySQL SELECT INTO OUTFILE statement. If I do: SELECT column1, column2 INTO OUTFILE 'outfile.csv' FIELDS TERMINATED BY ',' FROM table_name; outfile.csv will be created on the server in the same directory this database's files are stored in. However, when I change my query to: SELECT column1, column2 INTO OUTFILE '/data/outfile.csv' FIELDS TERMINATED BY ',' FROM table_name; I get: ERROR 1 (HY000): Can't create/write to file '/data/outfile.csv' (Errcode: 13) Errcode 13 is a permissions error, even if I change ownership of /data to mysql:mysql and give it 777 permissions. MySQL is running as user "mysql". Strangely, I can create the file in /tmp, just not in any other directory I've tried, even with permissions set such that user mysql should be able to write to the directory. This is MySQL 5.0.75 running on Ubuntu.

  Read the article

• MEB: Taking Incremental Backup using last successful backup

  - by Sagar Jauhari

  Introduction In MySQL Enterprise Backup v3.7.0 (MEB 3.7.0) a new option '–incremental-base' was introduced. Using this option a user can take in incremental backup without specifying the '–start-lsn' option. Description of this option can be found here. Instead of '–start-lsn' the user can provide the location of the last full backup or incremental backup using the 'dir:' prefix. MEB would extract the end LSN of this backup from the mysql.backup_history table as well as the backup_variables.txt file (for verification) to use it as the start LSN of the incremental backup. Because of popular demand, in MEB 3.7.1 the option '-incremental-base' has been extended further. The idea is to allow the user to take an incremental backup as easily as possible using the '–incremental-base' option. With the new option MEB queries the backup_history table for the last successful backup and uses its end LSN as the start LSN for the new incremental backup. It should be noted that the last successful backup is used irrespective of the location of the backup. Details A new prefix 'history:' has been introduced for the –incremental-base option and currently the only permissible value is the string "last_backup". So using the new option an incremental backup can be taken with the following command: $ mysqlbackup --incremental --incremental-backup-dir=/media/mysqlbackup-repo/ --incremental-base=history:last_backup backup When MEB attempts to extract the end LSN of the last successful backup from the mysql.backup_history table, it also scans the corresponding backup destination for the old backup and tries to read the meta files at this backup destination. If a valid backup still exists at the backup destination and the meta files can be read, MEB compares the end LSN found in the mysql.backup_history table with the end LSN found in the backup meta files of the old backup. Assuming that the host MySQL server is alive and mysql.backup_history can be accessed by MEB, the behaviour of MEB with respect to verification of the old end LSN can be summarized as follows: If 'BD' is the backup destination of the last successful backup in mysql.backup_history table and 'BHT' is the mysql.backup_history table if can_read_files_at_BD:     if end_lsn_found_at_BD == end_lsn_of_last_backup_in_BHT:         continue_with_backup()     else         return_with_error() else     continue_with_backup() Advantages Apart from ease of usability an important advantage of this option is that the user can do repeated incremental backups without changing the command line. This is possible using the '–with-timestamp' option along with this new option. For example, the following command $ mysqlbackup --with-timestamp --incremental --incremental-backup-dir=/media/mysqlbackup-repo/ --incremental-base=history:last_backup backup  can be used to perform successive incremental backups in the directory /media/mysqlbackup-repo . Limitations The option '--incremental-base=history:last_backup' should not be used when the user takes different kinds of concurrent backups on the same MySQL server (say different partial backups at multiple locations). should not be used after any temporary or experimental backups performed on the server (which where successful!). needs to be used with precaution since any intermediate successful backup without the –no-connection will be used as the base backup for the next incremental backup.  will give an error in case a valid backup exists at the location of the last successful backup and whose end LSN is different from that of the last successful backup found in the backup_history table. Date: 2012-06-19 HTML generated by org-mode 6.33x in emacs 23

  Read the article

• Cannot turn off autocommit in a script using the Django ORM

  - by Wes

  I have a command line script that uses the Django ORM and MySQL backend. I want to turn off autocommit and commit manually. For the life of me, I cannot get this to work. Here is a pared down version of the script. A row is inserted into testtable every time I run this and I get this warning from MySQL: "Some non-transactional changed tables couldn't be rolled back". #!/usr/bin/python import os import sys django_dir = os.path.abspath(os.path.normpath(os.path.join(os.path.dirname(__file__), '..'))) sys.path.append(django_dir) os.environ['DJANGO_DIR'] = django_dir os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings' from django.core.management import setup_environ from myproject import settings setup_environ(settings) from django.db import transaction, connection cursor = connection.cursor() cursor.execute('SET autocommit = 0') cursor.execute('insert into testtable values (\'X\')') cursor.execute('rollback') I also tried placing the insert in a function and adding Django's commit_manually wrapper, like so: @transaction.commit_manually def myfunction(): cursor = connection.cursor() cursor.execute('SET autocommit = 0') cursor.execute('insert into westest values (\'X\')') cursor.execute('rollback') myfunction() I also tried setting DISABLE_TRANSACTION_MANAGEMENT = True in settings.py, with no further luck. I feel like I am missing something obvious. Any help you can give me is greatly appreciated. Thanks!

  Read the article

• Python RSS Feeder + Jquery RSS Reader

  - by Panther24

  I'm trying to implement an RSS Feeder in python using PyRSS2Gen and it created an XML file. Now in jQuery I'm using Google Ajax API's RSS reader and trying to read it. I get an error saying something is wrong. I'm unable to figure out the issue. I've checked some documentation but not able to get it. This is how my xml file looks: <rss version="2.0"> <channel> <title>Andrew's PyRSS2Gen feed</title> <link> http://www.dalkescientific.com/Python/PyRSS2Gen.html </link> <description> The latest news about PyRSS2Gen, a Python library for generating RSS2 feeds </description> <lastBuildDate>Fri, 26 Mar 2010 13:10:55 GMT</lastBuildDate> <generator>PyRSS2Gen-1.0.0</generator> <docs>http://blogs.law.harvard.edu/tech/rss</docs> <item> <title>PyRSS2Gen-0.0 released</title> <link> http://www.dalkescientific.com/news/030906-PyRSS2Gen.html </link> <description> Dalke Scientific today announced PyRSS2Gen-0.0, a library for generating RSS feeds for Python. </description> <guid isPermaLink="true"> http://www.dalkescientific.com/news/030906-PyRSS2Gen.html </guid> <pubDate>Sat, 06 Sep 2003 21:31:00 GMT</pubDate> </item> <item> <title>Thoughts on RSS feeds for bioinformatics</title> <link> http://www.dalkescientific.com/writings/diary/archive/2003/09/06/RSS.html </link> <description> One of the reasons I wrote PyRSS2Gen was to experiment with RSS for data collection in bioinformatics. Last year I came across... </description> <guid isPermaLink="true"> http://www.dalkescientific.com/writings/diary/archive/2003/09/06/RSS.html </guid> <pubDate>Sat, 06 Sep 2003 21:49:00 GMT</pubDate> </item> </channel> </rss> And my JS looks like this <!-- ++Begin Dynamic Feed Wizard Generated Code++ --> <!-- // Created with a Google AJAX Search and Feed Wizard // http://code.google.com/apis/ajaxsearch/wizards.html --> <!-- // The Following div element will end up holding the actual feed control. // You can place this anywhere on your page. --> <!-- Google Ajax Api --> <script src="../js/jsapi.js" type="text/javascript"></script> <!-- Dynamic Feed Control and Stylesheet --> <script src="../js/gfdynamicfeedcontrol.js" type="text/javascript"></script> <link rel="stylesheet" type="text/css" href="../css/gfdynamicfeedcontrol.css"> <script type="text/javascript"> function LoadDynamicFeedControl() { var feeds = [ {title: 'Demo', url: 'wcarss.xml' }]; var options = { stacked : false, horizontal : false, title : "" } new GFdynamicFeedControl(feeds, 'feed-control', options); } // Load the feeds API and set the onload callback. google.load('feeds', '1'); google.setOnLoadCallback(LoadDynamicFeedControl); </script> <!-- ++End Dynamic Feed Control Wizard Generated Code++ --> Can some help me and tell me what I'm doing wrong here? NOTE: Does the XML file have to be deployed? Cause currently its in my local machine and I'm trying to just call it.

  Read the article

• Python TEA implementation

  - by Gaks

  Anybody knows proper python implementation of TEA (Tiny Encryption Algorithm)? I tried the one I've found here: http://sysadminco.com/code/python-tea/ - but it does not seem to work properly. It returns different results than other implementations in C or Java. I guess it's caused by completely different data types in python (or no data types in fact). Here's the code and an example: def encipher(v, k): y=v[0];z=v[1];sum=0;delta=0x9E3779B9;n=32 w=[0,0] while(n>0): y += (z << 4 ^ z >> 5) + z ^ sum + k[sum & 3] y &= 4294967295L # maxsize of 32-bit integer sum += delta z += (y << 4 ^ y >> 5) + y ^ sum + k[sum>>11 & 3] z &= 4294967295L n -= 1 w[0]=y; w[1]=z return w def decipher(v, k): y=v[0] z=v[1] sum=0xC6EF3720 delta=0x9E3779B9 n=32 w=[0,0] # sum = delta<<5, in general sum = delta * n while(n>0): z -= (y << 4 ^ y >> 5) + y ^ sum + k[sum>>11 & 3] z &= 4294967295L sum -= delta y -= (z << 4 ^ z >> 5) + z ^ sum + k[sum&3] y &= 4294967295L n -= 1 w[0]=y; w[1]=z return w Python example: >>> import tea >>> key = [0xbe168aa1, 0x16c498a3, 0x5e87b018, 0x56de7805] >>> v = [0xe15034c8, 0x260fd6d5] >>> res = tea.encipher(v, key) >>> "%X %X" % (res[0], res[1]) **'70D16811 F935148F'** C example: #include <unistd.h> #include <stdio.h> void encipher(unsigned long *const v,unsigned long *const w, const unsigned long *const k) { register unsigned long y=v[0],z=v[1],sum=0,delta=0x9E3779B9, a=k[0],b=k[1],c=k[2],d=k[3],n=32; while(n-->0) { sum += delta; y += (z << 4)+a ^ z+sum ^ (z >> 5)+b; z += (y << 4)+c ^ y+sum ^ (y >> 5)+d; } w[0]=y; w[1]=z; } int main() { unsigned long v[] = {0xe15034c8, 0x260fd6d5}; unsigned long key[] = {0xbe168aa1, 0x16c498a3, 0x5e87b018, 0x56de7805}; unsigned long res[2]; encipher(v, res, key); printf("%X %X\n", res[0], res[1]); return 0; } $ ./tea **D6942D68 6F87870D** Please note, that both examples were run with the same input data (v and key), but results were different. I'm pretty sure C implementation is correct - it comes from a site referenced by wikipedia (I couldn't post a link to it because I don't have enough reputation points yet - some antispam thing)

  Read the article

• Ubuntu: pip not working with python3.4

  - by val_

  Trying to get pip working on my Ubuntu pc. pip seems to be working for python2.7, but not for others. Here's the problem: $ pip Traceback (most recent call last): File "/usr/local/bin/pip", line 9, in <module> load_entry_point('pip==1.4.1', 'console_scripts', 'pip')() File "/usr/local/lib/python3.4/dist-packages/setuptools-1.1.5-py3.4.egg /pkg_resources.py", line 357, in load_entry_point def get_entry_info(dist, group, name): File "/usr/local/lib/python3.4/dist-packages/setuptools-1.1.5-py3.4.egg/pkg_resources.py", line 2394, in load_entry_point break File "/usr/local/lib/python3.4/dist-packages/setuptools-1.1.5-py3.4.egg/pkg_resources.py", line 2108, in load name = some.module:some.attr [extra1,extra2] ImportError: No module named 'pip' $ which pip /usr/local/bin/pip $ python2.7 -m pip //here can be just python, btw Usage: /usr/bin/python2.7 -m pip <command> [options] //and so on... $ python3.4 -m pip /usr/bin/python3.4: No module named pip From the home/user/.pip/pip.log : Exception: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main status = self.run(options, args) File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 283, in run requirement_set.install(install_options, global_options, root=options.root_path) File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1431, in install requirement.uninstall(auto_confirm=True) File "/usr/lib/python2.7/dist-packages/pip/req.py", line 598, in uninstall paths_to_remove.remove(auto_confirm) File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1836, in remove renames(path, new_path) File "/usr/lib/python2.7/dist-packages/pip/util.py", line 295, in renames shutil.move(old, new) File "/usr/lib/python2.7/shutil.py", line 303, in move os.unlink(src) OSError: [Errno 13] Permission denied: '/usr/bin/pip' There's no /usr/bin/pip btw. How can I fix this issue to work with pip and python 3.4 normally? I am trying to use pycharm, but it's package manager also stucks in this problem. Thanks for attention!

  Read the article

• Testing performance of queries in mysl

  - by Unreason

  I am trying to setup a script that would test performance of queries on a development mysql server. Here are more details: I have root access I am the only user accessing the server Mostly interested in InnoDB performance The queries I am optimizing are mostly search queries (SELECT ... LIKE '%xy%') What I want to do is to create reliable testing environment for measuring the speed of a single query, free from dependencies on other variables. Till now I have been using SQL_NO_CACHE, but sometimes the results of such tests also show caching behaviour - taking much longer to execute on the first run and taking less time on subsequent runs. If someone can explain this behaviour in full detail I might stick to using SQL_NO_CACHE; I do believe that it might be due to file system cache and/or caching of indexes used to execute the query, as this post explains. It is not clear to me when Buffer Pool and Key Buffer get invalidated or how they might interfere with testing. So, short of restarting mysql server, how would you recommend to setup an environment that would be reliable in determining if one query performs better then the other?

  Read the article

• MySQL query - if not exists - insert into - else - update

  - by user3180931

  I made a simple document generator by the form, this form saves everything to mysql database, It works great, but when someone type a the same 'nrumowy' it creates a new row in mysql, 'nrumowy' is unique, so when someone adds a form with the same 'nrumowy' I want to just update existing data in mysql, I have that code: $con=mysqli_connect("localhost","login","pass","database"); // Check connection if (mysqli_connect_errno()) { echo "Failed to connect to MySQL: " . mysqli_connect_error(); } // escape variables for security $numerklienta = mysqli_real_escape_string($con, $_POST['numerklienta']); $name = mysqli_real_escape_string($con, $_POST['name']); $hours = mysqli_real_escape_string($con, $_POST['hours']); $date = mysqli_real_escape_string($con, $_POST['date']); $beginDate = mysqli_real_escape_string($con, $_POST['beginDate']); $nrdomu = mysqli_real_escape_string($con, $_POST['nrdomu']); $telefon = mysqli_real_escape_string($con, $_POST['telefon']); $fax = mysqli_real_escape_string($con, $_POST['fax']); $nip = mysqli_real_escape_string($con, $_POST['nip']); $email = mysqli_real_escape_string($con, $_POST['email']); $stronawww = mysqli_real_escape_string($con, $_POST['stronawww']); $branza = mysqli_real_escape_string($con, $_POST['branza']); $vatkodpocztowy = mysqli_real_escape_string($con, $_POST['vatkodpocztowy']); $vatmiejscowosc = mysqli_real_escape_string($con, $_POST['vatmiejscowosc']); $vatulica = mysqli_real_escape_string($con, $_POST['vatulica']); $vatnrdomu = mysqli_real_escape_string($con, $_POST['vatnrdomu']); $vatemail = mysqli_real_escape_string($con, $_POST['vatemail']); $vatosoba = mysqli_real_escape_string($con, $_POST['vatosoba']); $datapublikacji = mysqli_real_escape_string($con, $_POST['datapublikacji']); $rabat = mysqli_real_escape_string($con, $_POST['rabat']); $wartoscnetto = mysqli_real_escape_string($con, $_POST['wartoscnetto']); $typreklamy = mysqli_real_escape_string($con, $_POST['typreklamy']); $inne = mysqli_real_escape_string($con, $_POST['inne']); $inne2 = mysqli_real_escape_string($con, $_POST['inne2']); $inne3 = mysqli_real_escape_string($con, $_POST['inne3']); $zaliczka = mysqli_real_escape_string($con, $_POST['zaliczka']); $liczbarat1 = mysqli_real_escape_string($con, $_POST['liczbarat1']); $zaakceptowaneprzez = mysqli_real_escape_string($con, $_POST['zaakceptowaneprzez']); $telzam = mysqli_real_escape_string($con, $_POST['telzam']); $datapodpis = mysqli_real_escape_string($con, $_POST['datapodpis']); $nrumowy = mysqli_real_escape_string($con, $_POST['nrumowy']); $sql="IF NOT EXISTS ( SELECT * FROM zam WHERE nrumowy = '$nrumowy' ) THEN INSERT INTO zam (numerklienta, name, hours, date, beginDate, nrdomu, telefon, fax, nip, email, stronawww, branza, vatkodpocztowy, vatmiejscowosc, vatulica, vatnrdomu, vatemail, vatosoba, datapublikacji, rabat, wartoscnetto, typreklamy, inne, inne2, inne3, zaliczka, liczbarat1, zaakceptowaneprzez, telzam, datapodpis, nrumowy) VALUES ('$numerklienta', '$name', '$hours', '$date', '$beginDate', '$nrdomu', '$telefon', '$fax', '$nip', '$email', '$stronawww', '$branza', '$vatkodpocztowy', '$vatmiejscowosc', '$vatulica', '$vatnrdomu', '$vatemail', '$vatosoba', '$datapublikacji', '$rabat', '$wartoscnetto', '$typreklamy', '$inne', '$inne2', '$inne3', '$zaliczka', '$liczbarat1', '$zaakceptowaneprzez', '$telzam', '$datapodpis', '$nrumowy' ) ELSE UPDATE zam SET name = '$name', numerklienta = '$numerklienta', hours = '$hours', date = '$date', beginDate = '$beginDate', nrdomu = '$nrdomu', telefon = '$telefon', fax = '$fax', nip = '$nip', email = '$email', stronawww = '$stronawww', branza = '$branza', vatkodpocztowy = '$vatkodpocztowy', vatmiejscowosc = '$vatmiejscowosc', vatulica = '$vatulica', vatnrdomu = '$vatnrdomu', vatemail = '$vatemail', vatosoba = '$vatosoba', datapublikacji = '$datapublikacji', rabat = '$rabat', wartoscnetto = '$wartoscnetto', typreklamy = '$typreklamy', inne = '$inne', inne2 = '$inne2', inne3 = '$inne3', zaliczka = '$zaliczka', liczbarat1 = '$liczbarat1', zaakceptowaneprzez = '$zaakceptowaneprzez', telzam = '$telzam', datapodpis = '$datapodpis' WHERE nrumowy ='$nrumowy' END IF"; if (!mysqli_query($con,$sql)) { die('Error: ' . mysqli_error($con)); } mysqli_close($con); This query without " select..... " and "else update" just a 'insert into' works great, also when I change this 'insert into' to 'update' but I don't know how to make this variable if not exists - insert into - else update

  Read the article

• Why are transactions not rolling back when using SpringJUnit4ClassRunner/MySQL/Spring/Hibernate

  - by Trevor

  I am doing unit testing and I expect that all data committed to the MySQL database will be rolled back... but this isn't the case. The data is being committed, even though my log was showing that the rollback was happening. I've been wrestling with this for a couple days so my setup has changed quite a bit, here's my current setup. LoginDAOTest.java: @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations={"file:web/WEB-INF/applicationContext-test.xml", "file:web/WEB-INF/dispatcher-servlet-test.xml"}) @TransactionConfiguration(transactionManager = "transactionManager", defaultRollback = true) public class UserServiceTest { private UserService userService; @Test public void should_return_true_when_user_is_logged_in () throws Exception { String[] usernames = {"a","b","c","d"}; for (String username : usernames) { userService.logUserIn(username); assertThat(userService.isUserLoggedIn(username), is(equalTo(true))); } } ApplicationContext-Text.xml: <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p" xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.5.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.5.xsd"> <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close"> <property name="driverClassName" value="com.mysql.jdbc.Driver"/> <property name="url" value="jdbc:mysql://localhost:3306/test"/> <property name="username" value="root"/> <property name="password" value="Ecosim07"/> </bean> <tx:annotation-driven transaction-manager="transactionManager"/> <bean id="userService" class="Service.UserService"> <property name="userDAO" ref="userDAO"/> </bean> <bean id="userDAO" class="DAO.UserDAO"> <property name="hibernateTemplate" ref="hibernateTemplate"/> </bean> <bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean"> <property name="dataSource" ref="dataSource"/> <property name="mappingResources"> <list> <value>/himapping/User.hbm.xml</value> <value>/himapping/setup.hbm.xml</value> <value>/himapping/UserHistory.hbm.xml</value> </list> </property> <property name="hibernateProperties"> <props> <prop key="hibernate.dialect">org.hibernate.dialect.SQLServerDialect</prop> <prop key="hibernate.show_sql">true</prop> </props> </property> </bean> <bean id="transactionManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager" p:sessionFactory-ref="sessionFactory"/> <bean id="hibernateTemplate" class="org.springframework.orm.hibernate3.HibernateTemplate"> <property name="sessionFactory"> <ref bean="sessionFactory"/> </property> </bean> </beans> I have been reading about the issue, and I've already checked to ensure that the MySQL database tables are setup to use InnoDB. Also I have been able to successfully implement rolling back of transactions outside of my testing suite. So this must be some sort of incorrect setup on my part. Any help would be greatly appreciated :)

  Read the article

• What should i do for accomodating large scale data storage and retrieval?

  - by kailashbuki

  There's two columns in the table inside mysql database. First column contains the fingerprint while the second one contains the list of documents which have that fingerprint. It's much like an inverted index built by search engines. An instance of a record inside the table is shown below; 34 "doc1, doc2, doc45" The number of fingerprints is very large(can range up to trillions). There are basically following operations in the database: inserting/updating the record & retrieving the record accoring to the match in fingerprint. The table definition python snippet is: self.cursor.execute("CREATE TABLE IF NOT EXISTS `fingerprint` (fp BIGINT, documents TEXT)") And the snippet for insert/update operation is: if self.cursor.execute("UPDATE `fingerprint` SET documents=CONCAT(documents,%s) WHERE fp=%s",(","+newDocId, thisFP))== 0L: self.cursor.execute("INSERT INTO `fingerprint` VALUES (%s, %s)", (thisFP,newDocId)) The only bottleneck i have observed so far is the query time in mysql. My whole application is web based. So time is a critical factor. I have also thought of using cassandra but have less knowledge of it. Please suggest me a better way to tackle this problem.

  Read the article

• Faster Insertion of Records into a Table with SQLAlchemy

  - by Kyle Brandt

  I am parsing a log and inserting it into either MySQL or SQLite using SQLAlchemy and Python. Right now I open a connection to the DB, and as I loop over each line, I insert it after it is parsed (This is just one big table right now, not very experienced with SQL). I then close the connection when the loop is done. The summarized code is: log_table = schema.Table('log_table', metadata, schema.Column('id', types.Integer, primary_key=True), schema.Column('time', types.DateTime), schema.Column('ip', types.String(length=15)) .... engine = create_engine(...) metadata.bind = engine connection = engine.connect() .... for line in file_to_parse: m = line_regex.match(line) if m: fields = m.groupdict() pythonified = pythoninfy_log(fields) #Turn them into ints, datatimes, etc if use_sql: ins = log_table.insert(values=pythonified) connection.execute(ins) parsed += 1 My two questions are: Is there a way to speed up the inserts within this basic framework? Maybe have a Queue of inserts and some insertion threads, some sort of bulk inserts, etc? When I used MySQL, for about ~1.2 million records the insert time was 15 minutes. With SQLite, the insert time was a little over an hour. Does that time difference between the db engines seem about right, or does it mean I am doing something very wrong?

  Read the article

• Configure Django project in a subdirectory using mod_python. Admin not working.

  - by David

  HI guys. I was trying to configure my django project in a subdirectory of the root, but didn't get things working.(LOcally it works perfect). I followed the django official django documentarion to deploy a project with mod_python. The real problem is that I am getting "Page not found" errors, whenever I try to go to the admin or any view of my apps. Here is my python.conf file located in /etc/httpd/conf.d/ in Fedora 7 LoadModule python_module modules/mod_python.so SetHandler python-program PythonHandler django.core.handlers.modpython SetEnv DJANGO_SETTINGS_MODULE mysite.settings PythonOption django.root /mysite PythonDebug On PythonPath "['/var/www/vhosts/mysite.com/httpdocs','/var/www/vhosts/mysite.com/httpdocs/mysite'] + sys.path" I know /var/www/ is not the best place to put my django project, but I just want to send a demo of my work in progress to my customer, later I will change the location. For example. If I go to www.domain.com/mysite/ I get the index view I configured in mysite.urls. But I cannot access to my app.urls (www.domain.com/mysite/app/) and any of the admin.urls.(www.domain.com/mysite/admin/) Here is mysite.urls: urlpatterns = patterns('', url(r'^admin/password_reset/$', 'django.contrib.auth.views.password_reset', name='password_reset'), (r'^password_reset/done/$', 'django.contrib.auth.views.password_reset_done'), (r'^reset/(?P<uidb36>[0-9A-Za-z]+)-(?P<token>.+)/$', 'django.contrib.auth.views.password_reset_confirm'), (r'^reset/done/$', 'django.contrib.auth.views.password_reset_complete'), (r'^$', 'app.views.index'), (r'^admin/', include(admin.site.urls)), (r'^app/', include('mysite.app.urls')), (r'^photologue/', include('photologue.urls')), ) I also tried changing admin.site.urls with ''django.contrib.admin.urls' , but it didn't worked. I googled a lot to solve this problem and read how other developers configure their django project, but didn't find too much information to deploy django in a subdirectory. I have the admin enabled in INSTALLED_APPS and the settings.py is ok. Please if you have any guide or telling me what I am doing wrong it will be much appreciated. THanks.

  Read the article

• Why does Akonadi on KDE 4.6.0 refuse to start?

  - by Patches

  Akonadi refuses to start on my fresh installation of KDE 4.6.0 from the kubuntu-backports PPA on Ubuntu 10.10 Maverick Meerkat, preventing me from usking KMail. Here is the full error output: patches@pleistocene:~/.local/share$ akonadictl start Starting Akonadi Server... done. patches@pleistocene:~/.local/share$ Connecting to deprecated signal QDBusConnectionInterface::serviceOwnerChanged(QString,QString,QString) search paths: ("/home/patches/bin", "/usr/local/sbin", "/usr/local/bin", "/usr/sbin", "/usr/bin", "/sbin", "/bin", "/usr/games", "/usr/sbin", "/usr/local/sbin", "/usr/local/libexec", "/usr/libexec", "/opt/mysql/libexec", "/opt/local/lib/mysql5/bin", "/opt/mysql/sbin") Found mysql_install_db: "/usr/bin/mysql_install_db" Found mysqlcheck: "/usr/bin/mysqlcheck" Database process exited unexpectedly during initial connection! executable: "/usr/sbin/mysqld-akonadi" arguments: ("--defaults-file=/home/patches/.local/share/akonadi//mysql.conf", "--datadir=/home/patches/.local/share/akonadi/db_data/", "--socket=/home/patches/.local/share/akonadi/socket-pleistocene/mysql.socket") stdout: "" stderr: "Could not open required defaults file: /home/patches/.local/share/akonadi//mysql.conf Fatal error in defaults handling. Program aborted 110209 16:41:12 [Warning] Can't create test file /home/patches/.local/share/akonadi/db_data/pleistocene.lower-test 110209 16:41:12 [Warning] Can't create test file /home/patches/.local/share/akonadi/db_data/pleistocene.lower-test 110209 16:41:12 [Note] Plugin 'FEDERATED' is disabled. /usr/sbin/mysqld-akonadi: Can't find file: './mysql/plugin.frm' (errno: 13) 110209 16:41:12 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it. 110209 16:41:12 InnoDB: Operating system error number 13 in a file operation. InnoDB: The error means mysqld does not have the access rights to InnoDB: the directory. InnoDB: File name ./ibdata1 InnoDB: File operation call: 'create'. InnoDB: Cannot continue operation. " exit code: 1 process error: "Unknown error" "[ 0: akonadiserver(_Z11akBacktracev+0x35) [0x8086055] 1: akonadiserver() [0x8086516] 2: [0xb772e400] 3: [0xb772e416] 4: /lib/libc.so.6(gsignal+0x51) [0xb6e9f941] 5: /lib/libc.so.6(abort+0x182) [0xb6ea2e42] 6: /usr/lib/libQtCore.so.4(_Z17qt_message_output9QtMsgTypePKc+0x8c) [0xb74d62dc] 7: akonadiserver(_ZN15FileDebugStream9writeDataEPKcx+0xc4) [0x8087574] 8: /usr/lib/libQtCore.so.4(_ZN9QIODevice5writeEPKcx+0x8e) [0xb757168e] 9: /usr/lib/libQtCore.so.4(+0x103425) [0xb7581425] 10: /usr/lib/libQtCore.so.4(_ZN11QTextStreamD1Ev+0x3d) [0xb758295d] 11: akonadiserver(_ZN6QDebugD1Ev+0x43) [0x8081b73] 12: akonadiserver(_ZN13DbConfigMysql19startInternalServerEv+0x1c27) [0x810c177] 13: akonadiserver(_ZN7Akonadi13AkonadiServer20startDatabaseProcessEv+0xe3) [0x8087a23] 14: akonadiserver(_ZN7Akonadi13AkonadiServerC1EP7QObject+0xca) [0x8088b6a] 15: akonadiserver(_ZN7Akonadi13AkonadiServer8instanceEv+0x48) [0x808a1d8] 16: akonadiserver(main+0x364) [0x8080fb4] 17: /lib/libc.so.6(__libc_start_main+0xe7) [0xb6e8bce7] 18: akonadiserver() [0x8080b81] ] " ProcessControl: Application 'akonadiserver' returned with exit code 255 (Unknown error) search paths: ("/home/patches/bin", "/usr/local/sbin", "/usr/local/bin", "/usr/sbin", "/usr/bin", "/sbin", "/bin", "/usr/games", "/usr/sbin", "/usr/local/sbin", "/usr/local/libexec", "/usr/libexec", "/opt/mysql/libexec", "/opt/local/lib/mysql5/bin", "/opt/mysql/sbin") Found mysql_install_db: "/usr/bin/mysql_install_db" Found mysqlcheck: "/usr/bin/mysqlcheck" Database process exited unexpectedly during initial connection! executable: "/usr/sbin/mysqld-akonadi" arguments: ("--defaults-file=/home/patches/.local/share/akonadi//mysql.conf", "--datadir=/home/patches/.local/share/akonadi/db_data/", "--socket=/home/patches/.local/share/akonadi/socket-pleistocene/mysql.socket") stdout: "" stderr: "Could not open required defaults file: /home/patches/.local/share/akonadi//mysql.conf Fatal error in defaults handling. Program aborted 110209 16:41:12 [Warning] Can't create test file /home/patches/.local/share/akonadi/db_data/pleistocene.lower-test 110209 16:41:12 [Warning] Can't create test file /home/patches/.local/share/akonadi/db_data/pleistocene.lower-test 110209 16:41:12 [Note] Plugin 'FEDERATED' is disabled. /usr/sbin/mysqld-akonadi: Can't find file: './mysql/plugin.frm' (errno: 13) 110209 16:41:12 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it. 110209 16:41:12 InnoDB: Operating system error number 13 in a file operation. InnoDB: The error means mysqld does not have the access rights to InnoDB: the directory. InnoDB: File name ./ibdata1 InnoDB: File operation call: 'create'. InnoDB: Cannot continue operation. " exit code: 1 process error: "Unknown error" "[ 0: akonadiserver(_Z11akBacktracev+0x35) [0x8086055] 1: akonadiserver() [0x8086516] 2: [0xb77ae400] 3: [0xb77ae416] 4: /lib/libc.so.6(gsignal+0x51) [0xb6f1f941] 5: /lib/libc.so.6(abort+0x182) [0xb6f22e42] 6: /usr/lib/libQtCore.so.4(_Z17qt_message_output9QtMsgTypePKc+0x8c) [0xb75562dc] 7: akonadiserver(_ZN15FileDebugStream9writeDataEPKcx+0xc4) [0x8087574] 8: /usr/lib/libQtCore.so.4(_ZN9QIODevice5writeEPKcx+0x8e) [0xb75f168e] 9: /usr/lib/libQtCore.so.4(+0x103425) [0xb7601425] 10: /usr/lib/libQtCore.so.4(_ZN11QTextStreamD1Ev+0x3d) [0xb760295d] 11: akonadiserver(_ZN6QDebugD1Ev+0x43) [0x8081b73] 12: akonadiserver(_ZN13DbConfigMysql19startInternalServerEv+0x1c27) [0x810c177] 13: akonadiserver(_ZN7Akonadi13AkonadiServer20startDatabaseProcessEv+0xe3) [0x8087a23] 14: akonadiserver(_ZN7Akonadi13AkonadiServerC1EP7QObject+0xca) [0x8088b6a] 15: akonadiserver(_ZN7Akonadi13AkonadiServer8instanceEv+0x48) [0x808a1d8] 16: akonadiserver(main+0x364) [0x8080fb4] 17: /lib/libc.so.6(__libc_start_main+0xe7) [0xb6f0bce7] 18: akonadiserver() [0x8080b81] ] " ProcessControl: Application 'akonadiserver' returned with exit code 255 (Unknown error) search paths: ("/home/patches/bin", "/usr/local/sbin", "/usr/local/bin", "/usr/sbin", "/usr/bin", "/sbin", "/bin", "/usr/games", "/usr/sbin", "/usr/local/sbin", "/usr/local/libexec", "/usr/libexec", "/opt/mysql/libexec", "/opt/local/lib/mysql5/bin", "/opt/mysql/sbin") Found mysql_install_db: "/usr/bin/mysql_install_db" Found mysqlcheck: "/usr/bin/mysqlcheck" Database process exited unexpectedly during initial connection! executable: "/usr/sbin/mysqld-akonadi" arguments: ("--defaults-file=/home/patches/.local/share/akonadi//mysql.conf", "--datadir=/home/patches/.local/share/akonadi/db_data/", "--socket=/home/patches/.local/share/akonadi/socket-pleistocene/mysql.socket") stdout: "" stderr: "Could not open required defaults file: /home/patches/.local/share/akonadi//mysql.conf Fatal error in defaults handling. Program aborted 110209 16:41:12 [Warning] Can't create test file /home/patches/.local/share/akonadi/db_data/pleistocene.lower-test 110209 16:41:12 [Warning] Can't create test file /home/patches/.local/share/akonadi/db_data/pleistocene.lower-test 110209 16:41:12 [Note] Plugin 'FEDERATED' is disabled. /usr/sbin/mysqld-akonadi: Can't find file: './mysql/plugin.frm' (errno: 13) 110209 16:41:12 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it. 110209 16:41:12 InnoDB: Operating system error number 13 in a file operation. InnoDB: The error means mysqld does not have the access rights to InnoDB: the directory. InnoDB: File name ./ibdata1 InnoDB: File operation call: 'create'. InnoDB: Cannot continue operation. " exit code: 1 process error: "Unknown error" "[ 0: akonadiserver(_Z11akBacktracev+0x35) [0x8086055] 1: akonadiserver() [0x8086516] 2: [0xb778b400] 3: [0xb778b416] 4: /lib/libc.so.6(gsignal+0x51) [0xb6efc941] 5: /lib/libc.so.6(abort+0x182) [0xb6effe42] 6: /usr/lib/libQtCore.so.4(_Z17qt_message_output9QtMsgTypePKc+0x8c) [0xb75332dc] 7: akonadiserver(_ZN15FileDebugStream9writeDataEPKcx+0xc4) [0x8087574] 8: /usr/lib/libQtCore.so.4(_ZN9QIODevice5writeEPKcx+0x8e) [0xb75ce68e] 9: /usr/lib/libQtCore.so.4(+0x103425) [0xb75de425] 10: /usr/lib/libQtCore.so.4(_ZN11QTextStreamD1Ev+0x3d) [0xb75df95d] 11: akonadiserver(_ZN6QDebugD1Ev+0x43) [0x8081b73] 12: akonadiserver(_ZN13DbConfigMysql19startInternalServerEv+0x1c27) [0x810c177] 13: akonadiserver(_ZN7Akonadi13AkonadiServer20startDatabaseProcessEv+0xe3) [0x8087a23] 14: akonadiserver(_ZN7Akonadi13AkonadiServerC1EP7QObject+0xca) [0x8088b6a] 15: akonadiserver(_ZN7Akonadi13AkonadiServer8instanceEv+0x48) [0x808a1d8] 16: akonadiserver(main+0x364) [0x8080fb4] 17: /lib/libc.so.6(__libc_start_main+0xe7) [0xb6ee8ce7] 18: akonadiserver() [0x8080b81] ] " ProcessControl: Application 'akonadiserver' returned with exit code 255 (Unknown error) search paths: ("/home/patches/bin", "/usr/local/sbin", "/usr/local/bin", "/usr/sbin", "/usr/bin", "/sbin", "/bin", "/usr/games", "/usr/sbin", "/usr/local/sbin", "/usr/local/libexec", "/usr/libexec", "/opt/mysql/libexec", "/opt/local/lib/mysql5/bin", "/opt/mysql/sbin") Found mysql_install_db: "/usr/bin/mysql_install_db" Found mysqlcheck: "/usr/bin/mysqlcheck" Database process exited unexpectedly during initial connection! executable: "/usr/sbin/mysqld-akonadi" arguments: ("--defaults-file=/home/patches/.local/share/akonadi//mysql.conf", "--datadir=/home/patches/.local/share/akonadi/db_data/", "--socket=/home/patches/.local/share/akonadi/socket-pleistocene/mysql.socket") stdout: "" stderr: "Could not open required defaults file: /home/patches/.local/share/akonadi//mysql.conf Fatal error in defaults handling. Program aborted 110209 16:41:12 [Warning] Can't create test file /home/patches/.local/share/akonadi/db_data/pleistocene.lower-test 110209 16:41:12 [Warning] Can't create test file /home/patches/.local/share/akonadi/db_data/pleistocene.lower-test 110209 16:41:12 [Note] Plugin 'FEDERATED' is disabled. /usr/sbin/mysqld-akonadi: Can't find file: './mysql/plugin.frm' (errno: 13) 110209 16:41:12 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it. 110209 16:41:12 InnoDB: Operating system error number 13 in a file operation. InnoDB: The error means mysqld does not have the access rights to InnoDB: the directory. InnoDB: File name ./ibdata1 InnoDB: File operation call: 'create'. InnoDB: Cannot continue operation. " exit code: 1 process error: "Unknown error" "[ 0: akonadiserver(_Z11akBacktracev+0x35) [0x8086055] 1: akonadiserver() [0x8086516] 2: [0xb784e400] 3: [0xb784e416] 4: /lib/libc.so.6(gsignal+0x51) [0xb6fbf941] 5: /lib/libc.so.6(abort+0x182) [0xb6fc2e42] 6: /usr/lib/libQtCore.so.4(_Z17qt_message_output9QtMsgTypePKc+0x8c) [0xb75f62dc] 7: akonadiserver(_ZN15FileDebugStream9writeDataEPKcx+0xc4) [0x8087574] 8: /usr/lib/libQtCore.so.4(_ZN9QIODevice5writeEPKcx+0x8e) [0xb769168e] 9: /usr/lib/libQtCore.so.4(+0x103425) [0xb76a1425] 10: /usr/lib/libQtCore.so.4(_ZN11QTextStreamD1Ev+0x3d) [0xb76a295d] 11: akonadiserver(_ZN6QDebugD1Ev+0x43) [0x8081b73] 12: akonadiserver(_ZN13DbConfigMysql19startInternalServerEv+0x1c27) [0x810c177] 13: akonadiserver(_ZN7Akonadi13AkonadiServer20startDatabaseProcessEv+0xe3) [0x8087a23] 14: akonadiserver(_ZN7Akonadi13AkonadiServerC1EP7QObject+0xca) [0x8088b6a] 15: akonadiserver(_ZN7Akonadi13AkonadiServer8instanceEv+0x48) [0x808a1d8] 16: akonadiserver(main+0x364) [0x8080fb4] 17: /lib/libc.so.6(__libc_start_main+0xe7) [0xb6fabce7] 18: akonadiserver() [0x8080b81] ] " ProcessControl: Application 'akonadiserver' returned with exit code 255 (Unknown error) "akonadiserver" crashed too often and will not be restarted! I tried moving the ~/.local/share/akonadi folder and running it fresh, and I also tried starting Akonadi from a brand new user, all to no avail. Requested by @djeikyb: patches@pleistocene:~$ ls -ld ~/.local drwxrwx--- 3 patches patches 4096 2011-02-07 03:15 /home/patches/.local patches@pleistocene:~$ mysql_upgrade Looking for 'mysql' as: mysql Looking for 'mysqlcheck' as: mysqlcheck Running 'mysqlcheck' with connection arguments: '--port=3306' '--socket=/var/run/mysqld/mysqld.sock' mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) when trying to connect FATAL ERROR: Upgrade failed patches@pleistocene:~$ mysql_upgrade -S ~/.local/share/akonadi/socket-pleistocene/ Looking for 'mysql' as: mysql Looking for 'mysqlcheck' as: mysqlcheck Running 'mysqlcheck' with connection arguments: '--port=3306' '--socket=/var/run/mysqld/mysqld.sock' '--socket=/home/patches/.local/share/akonadi/socket-pleistocene/' mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/home/patches/.local/share/akonadi/socket-pleistocene/' (111) when trying to connect FATAL ERROR: Upgrade failed

  Read the article

• Solving embarassingly parallel problems using Python multiprocessing

  - by gotgenes

  How does one use multiprocessing to tackle embarrassingly parallel problems? Embarassingly parallel problems typically consist of three basic parts: Read input data (from a file, database, tcp connection, etc.). Run calculations on the input data, where each calculation is independent of any other calculation. Write results of calculations (to a file, database, tcp connection, etc.). We can parallelize the program in two dimensions: Part 2 can run on multiple cores, since each calculation is independent; order of processing doesn't matter. Each part can run independently. Part 1 can place data on an input queue, part 2 can pull data off the input queue and put results onto an output queue, and part 3 can pull results off the output queue and write them out. This seems a most basic pattern in concurrent programming, but I am still lost in trying to solve it, so let's write a canonical example to illustrate how this is done using multiprocessing. Here is the example problem: Given a CSV file with rows of integers as input, compute their sums. Separate the problem into three parts, which can all run in parallel: Process the input file into raw data (lists/iterables of integers) Calculate the sums of the data, in parallel Output the sums Below is traditional, single-process bound Python program which solves these three tasks: #!/usr/bin/env python # -*- coding: UTF-8 -*- # basicsums.py """A program that reads integer values from a CSV file and writes out their sums to another CSV file. """ import csv import optparse import sys def make_cli_parser(): """Make the command line interface parser.""" usage = "\n\n".join(["python %prog INPUT_CSV OUTPUT_CSV", __doc__, """ ARGUMENTS: INPUT_CSV: an input CSV file with rows of numbers OUTPUT_CSV: an output file that will contain the sums\ """]) cli_parser = optparse.OptionParser(usage) return cli_parser def parse_input_csv(csvfile): """Parses the input CSV and yields tuples with the index of the row as the first element, and the integers of the row as the second element. The index is zero-index based. :Parameters: - `csvfile`: a `csv.reader` instance """ for i, row in enumerate(csvfile): row = [int(entry) for entry in row] yield i, row def sum_rows(rows): """Yields a tuple with the index of each input list of integers as the first element, and the sum of the list of integers as the second element. The index is zero-index based. :Parameters: - `rows`: an iterable of tuples, with the index of the original row as the first element, and a list of integers as the second element """ for i, row in rows: yield i, sum(row) def write_results(csvfile, results): """Writes a series of results to an outfile, where the first column is the index of the original row of data, and the second column is the result of the calculation. The index is zero-index based. :Parameters: - `csvfile`: a `csv.writer` instance to which to write results - `results`: an iterable of tuples, with the index (zero-based) of the original row as the first element, and the calculated result from that row as the second element """ for result_row in results: csvfile.writerow(result_row) def main(argv): cli_parser = make_cli_parser() opts, args = cli_parser.parse_args(argv) if len(args) != 2: cli_parser.error("Please provide an input file and output file.") infile = open(args[0]) in_csvfile = csv.reader(infile) outfile = open(args[1], 'w') out_csvfile = csv.writer(outfile) # gets an iterable of rows that's not yet evaluated input_rows = parse_input_csv(in_csvfile) # sends the rows iterable to sum_rows() for results iterable, but # still not evaluated result_rows = sum_rows(input_rows) # finally evaluation takes place as a chain in write_results() write_results(out_csvfile, result_rows) infile.close() outfile.close() if __name__ == '__main__': main(sys.argv[1:]) Let's take this program and rewrite it to use multiprocessing to parallelize the three parts outlined above. Below is a skeleton of this new, parallelized program, that needs to be fleshed out to address the parts in the comments: #!/usr/bin/env python # -*- coding: UTF-8 -*- # multiproc_sums.py """A program that reads integer values from a CSV file and writes out their sums to another CSV file, using multiple processes if desired. """ import csv import multiprocessing import optparse import sys NUM_PROCS = multiprocessing.cpu_count() def make_cli_parser(): """Make the command line interface parser.""" usage = "\n\n".join(["python %prog INPUT_CSV OUTPUT_CSV", __doc__, """ ARGUMENTS: INPUT_CSV: an input CSV file with rows of numbers OUTPUT_CSV: an output file that will contain the sums\ """]) cli_parser = optparse.OptionParser(usage) cli_parser.add_option('-n', '--numprocs', type='int', default=NUM_PROCS, help="Number of processes to launch [DEFAULT: %default]") return cli_parser def main(argv): cli_parser = make_cli_parser() opts, args = cli_parser.parse_args(argv) if len(args) != 2: cli_parser.error("Please provide an input file and output file.") infile = open(args[0]) in_csvfile = csv.reader(infile) outfile = open(args[1], 'w') out_csvfile = csv.writer(outfile) # Parse the input file and add the parsed data to a queue for # processing, possibly chunking to decrease communication between # processes. # Process the parsed data as soon as any (chunks) appear on the # queue, using as many processes as allotted by the user # (opts.numprocs); place results on a queue for output. # # Terminate processes when the parser stops putting data in the # input queue. # Write the results to disk as soon as they appear on the output # queue. # Ensure all child processes have terminated. # Clean up files. infile.close() outfile.close() if __name__ == '__main__': main(sys.argv[1:]) These pieces of code, as well as another piece of code that can generate example CSV files for testing purposes, can be found on github. I would appreciate any insight here as to how you concurrency gurus would approach this problem. Here are some questions I had when thinking about this problem. Bonus points for addressing any/all: Should I have child processes for reading in the data and placing it into the queue, or can the main process do this without blocking until all input is read? Likewise, should I have a child process for writing the results out from the processed queue, or can the main process do this without having to wait for all the results? Should I use a processes pool for the sum operations? If yes, what method do I call on the pool to get it to start processing the results coming into the input queue, without blocking the input and output processes, too? apply_async()? map_async()? imap()? imap_unordered()? Suppose we didn't need to siphon off the input and output queues as data entered them, but could wait until all input was parsed and all results were calculated (e.g., because we know all the input and output will fit in system memory). Should we change the algorithm in any way (e.g., not run any processes concurrently with I/O)?

  Read the article

• Using tinybutstrong templating system, how do I have a (secondary) mysql query (sub-block), inside a

  - by desbest

  This is the code that works well. $TBS->MergeBlock("items",$conn,"SELECT * FROM items WHERE categoryid = $getcategory"); and it has a good job of looping the items with no problem. <tr id="itemid_[items.id;block=tr]"> <td height="30"> <!-- <img src="move.png" align="left" style="margin-right: 8px;"> --> <div class="lowlight">[items.title] </div> </td> <td height="30"> <div class="editable" itemid="$item[id]">[items.quantity]</div> </td> <td height="30"> <!-- <img src="pencil.png" id="edititem"width="24" height="24"> --> &nbsp; &nbsp; &nbsp; &nbsp;<img src="icons/folder.png" id="category_[items.categoryid]";" class="changecategory" categoryid="[items.categoryid]" itemid="[items.id]" itemtitle="[items.title]" title="Change category" width="24" height="24"> &nbsp; &nbsp; &nbsp; &nbsp; <a href="index.php?category=[var.getcategory]&deleteitem=[items.id]" title="Delete item" onclick="return confirm('Are you sure you want to delete [items.title]?')"><img src="icons/trash.png" id="deleteitem" width="24" height="24"></a> </td> </tr> This is where the problem lies. In the table row I would like a secondary mysql merge block based on the id column that the primary (items) mysql table has. This means that ideally I would love to do this. $TBS->MergeBlock("fields",$conn,"SELECT * FROM items WHERE categoryid = [items.id]"); So then when I used [fields.value;block=div] inside the table row, it would pick up the a row from the fields table based on the primary mysql query. It would recognise what the id is for the merged block and perform a mysql query based on a variable that that block has. So that's what I would like to do and that's what I call a secondary mysql merge block. This is what I attempted by using the sub-blocks example and modifying it. index.html <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <title>TinyButStrong - Examples - subblocks</title> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <link href="tbs_us_examples_0styles.css" rel="stylesheet" type="text/css"> <style type="text/css"> <!-- .border02 { border: 2px solid #39C; } .border03 { border: 2px solid #396; } --> </style> </head> <body> <table width="400" align="center" cellpadding="5" cellspacing="0" style="border-collapse:collapse;"> <tr> <td class="border02"><strong>Item name:</strong> [item.title;block=tr] , <strong>Item quantity:</strong> [item.quantity]<br> <table border="1" align="center" cellpadding="2" cellspacing="0"> <tr bgcolor="#CACACA"> <td width="30"><u>Position</u></td> <td width="150"><u>Attribute</u></td> <td width="50"><u>Value</u></td> <td width="100"><div align="center"><u>Date Number</u></div></td> </tr> <tr bgcolor="#F0F0F0"> <td>[field.#]</td> <td>[field.attribute;block=tr;p1=[item.id]] <br><b>[item.id]</b> </td> <td><div align="right">[field.value]</div></td> <td><div align="center">[field.datenumber;frm='mm-dd-yyyy']</div></td> </tr> </table> </td> </tr> </table> </body> </html> index.php <?php require_once('../stockman-v3/tbs_class.php'); require_once('../stockman-v3/config.php'); // Create data $TeamList[0] = array('team'=>'Eagle' ,'total'=>'458'); //{ $TeamList[0]['matches'][] = array('town'=>'London','score'=>'253','date'=>'1999-11-30'); $TeamList[0]['matches'][] = array('town'=>'Paris' ,'score'=>'145','date'=>'2002-07-24'); $TeamList[1] = array('team'=>'Goonies','total'=>'281'); $TeamList[1]['matches'][] = array('town'=>'New-York','score'=>'365','date'=>'2001-12-25'); $TeamList[1]['matches'][] = array('town'=>'Madrid' ,'score'=>'521','date'=>'2004-01-14'); // } $TeamList[2] = array('team'=>'MIB' ,'total'=>'615'); // { $TeamList[2]['matches'][] = array('town'=>'Dallas' ,'score'=>'362','date'=>'2001-01-02'); $TeamList[2]['matches'][] = array('town'=>'Lyon' ,'score'=>'321','date'=>'2002-11-17'); $TeamList[2]['matches'][] = array('town'=>'Washington','score'=>'245','date'=>'2003-08-24'); // } $TBS = new clsTinyButStrong; $TBS->LoadTemplate('index3.html'); // Automatic subblock $TBS->MergeBlock("item",$conn,"SELECT * FROM items"); $TBS->MergeBlock("field",$conn,"SELECT * FROM fields WHERE itemid='[%p1%]' "); $TBS->MergeBlock('teamKEY',$TeamList); // Subblock with a dynamic query //$TBS->MergeBlock('match','array','TeamList[%p1%][matches]'); $TBS->MergeBlock("match",$conn,"SELECT * FROM fields WHERE id='[%p1%]' "); $TBS->Show(); ?> please note what i am trying to do If I was to change $TBS->MergeBlock("field",$conn,"SELECT * FROM fields WHERE itemid='[%p1%]' "); to... $TBS->MergeBlock("field",$conn,"SELECT * FROM fields"); It would show all the items.

  Read the article

• how to define service runlevel order position?

  - by DmitrySemenov

  I setup bind-dlz and need mysql start prior NAMED when system starts here is what I have [root@semenov]# ./test.sh mysql 0:off 1:off 2:on 3:on 4:on 5:on 6:off named 0:off 1:off 2:off 3:on 4:on 5:on 6:off lrwxrwxrwx. 1 root root 15 Apr 15 18:57 /etc/rc3.d/S93mysql -> ../init.d/mysql lrwxrwxrwx. 1 root root 15 Apr 15 18:57 /etc/rc3.d/S90named -> ../init.d/named here is what I have in mysql init script # Comments to support chkconfig on RedHat Linux # chkconfig: 2345 84 16 # description: A very fast and reliable SQL database engine. # Comments to support LSB init script conventions ### BEGIN INIT INFO # Provides: mysql # Required-Start: $local_fs $network $remote_fs # Should-Start: ypbind nscd ldap ntpd xntpd # Required-Stop: $local_fs $network $remote_fs # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: start and stop MySQL # Description: MySQL is a very fast and reliable SQL database engine. ### END INIT INFO so when I remove named from chkconfig and have there just mysql, it starts with order number 84: /etc/rc3.d/S84mysql - ../init.d/mysql but when I add named inside chkconfig it's order changes to 93: /etc/rc3.d/S93mysql - ../init.d/mysql as a result mysql will be starting after named and named will fail (no sql available) any ideas what I'm doing wrong? here is what I have in named init script # chkconfig: 345 90 16 # description: named (BIND) is a Domain Name Server (DNS) \ # that is used to resolve host names to IP addresses. # probe: true ### BEGIN INIT INFO # Provides: $named # Required-Start: $local_fs $network $syslog # Required-Stop: $local_fs $network $syslog # Default-Start:2 3 4 # Default-Stop: 0 1 2 3 4 5 6 # Short-Description: start|stop|status|restart|try-restart|reload|force-reload DNS server # Description: control ISC BIND implementation of DNS server ### END INIT INFO thanks, Dmitry

  Read the article

• Count Email Address Domains

  - by BRADINO

  A quick tidbit I came up with today to count email addresses in a mysql database table grouping them by domain. So say for example you have a large list of subscribers and you want to see the breakdown of people who use Hotmail, Yahoo, Gmail, etc. SELECT COUNT( SUBSTRING_INDEX( `email` , '@', -1 ) ) AS `count` , SUBSTRING_INDEX( `email` , '@', -1 ) AS `domain` FROM `subscribers` WHERE `email` != '' GROUP BY `domain` ORDER BY `count` DESC This sql statement assumes that the table is called 'subscribers' and the column containing the email addresses is 'email'. Change these two values to match your table name and email address column name. mysql count email mysql count domain mysql split email mysql split domain

  Read the article

• Any good tutorials all this web programming stuff for a GUI person? [closed]

  - by supercheetah

  For some reason, I am having a hard time understanding all this web programming stuff--from AJAX to JSON, etc. I've got plenty of experience programming GUIs. I'm currently working on a project in Python, and I thought that maybe I could just use PyJS (since it's GWT for Python, it uses an API that's very familiar to experienced GUI programmers like myself) to compile it with a Javascript interface on top, but alas, the compiler gave me a spectacular failure. It's obviously not meant to handle much of any Python beyond itself, and some of the core Python library. It would have been nice if it could, but I will admit, it would have been the lazy way to do it. I tried to learn Django, but for some reason, I'm just having a hard time understanding the tutorial on their website, and what it's all doing. Maybe it's not the best framework to learn, perhaps? Anyway, does anyone have a good primer/tutorial explaining all this stuff, especially for Python, and especially for someone coming from a GUI background?

  Read the article

• Use the latest technology or use a mature technology as a developer?

  - by Ted Wong

  I would like to develop an application for a group of people to use. I have decided to develop using python, but I am thinking of using python 2.X or python 3.X. If I use python 2.X, I need to upgrade it for the future... But it is more mature, and has many tools and libraries. If I develop using 3.X, I don't need to think of future integration, but currenttly it doesn't have many libraries, even a python to executable is not ready for all platforms. Also, one of the considerations is that it is a brand new application, so I don't have the history burden to maintain the old libraries. Any recommendation on this dilemma? More information about this application: Native application Time for maintenance: 5 years+ Library/Tools must need: don't have idea, yet. Must need feature that in 2.X: Convert to an executable for both Windows and Mac OS X

  Read the article

• How to execute? [closed]

  - by Viswa

  Possible Duplicate: how to read the password from variable? I did the below code in my python script,but its not work. #! /usr/bin/python import os address = "rsync -avrz [email protected]:/opt/script/python/data/ /opt/script/python/data/" passwd ="my server password" os.system('%(address)s "echo %(passwd)s"' %locals()) it throws below error. If arg is a remote file/dir, prefix it with a colon (:). rsync error: syntax or usage error (code 1) at main.c(1236) [Receiver=3.0.7] If i run os.system('%(address)s' %locals()) means it work without any error but it ask password. I need that password should be read from my passwd variable. How to write python script to read server password from my variable.

  Read the article

• Why does IDLE continue to crash? [migrated]

  - by Dyana

  Idle keeps crashing and I can't figure it out. After restarting the computer and reinstalling Python, none of which seemed to work, I looked to my peers and was told to "install one of the Tcl/Tk". After getting another opinion I was also told that I already had this and found it to be true but decided to try it anyway since it continued to crash. Nothing has improved and I have an assignment due. Any ideas on why this continues to happen and what I can do to fix the crash? Problem details: Process: Python [1183] Path: /Applications/Python 3.3/IDLE.app/Contents/MacOS/Python Identifier: org.python.IDLE Version: 3.3.0 (3.3.0) Code Type: X86-64 (Native) Parent Process: launchd [793] Date/Time: 2012-11-05 14:10:54.124 -0500 OS Version: Mac OS X 10.7.5 (11G63) Report Version: 9 Interval Since Last Report: 181805 sec Crashes Since Last Report: 4 Per-App Interval Since Last Report: 20 sec Per-App Crashes Since Last Report: 4 Anonymous UUID: 68994A08-7FFB-4074-A553-CB60A60BB412 Crashed Thread: 0 Dispatch queue: com.apple.main-thread Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Application Specific Information: * Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Error (1007) creating CGSWindow on line 263'

  Read the article

• mayavi2 installing has problem (ubuntu 12.04)

  - by user98865

  I'm using ubuntu 12.04 and python 2.7.3-0ubuntu2 is already installed. I have a problem during installing mayavi2 from ubuntu software center. Error message is : Package dependencies cannot be resolved This error could be caused by required additional software packages which are missing or not installable. Furthermore there could be a conflict between software packages which are not allowed to be installed at the same time. Details: The following packages have unmet dependencies mayavi2: Depends: python-numpy (= 1:1.6.1) but 1:1.6.1-6ubuntu1 is to be installed Depends: python-numpy-abi9 but it is a virtual package Depends: python (< 2.8) but 2.7.3-0ubuntu2 is to be installed Depends: python-vtk (= 5.4.2-5) but 5.8.0-5 is to be installed I've searched to solve this problem for a long time but I didn't solve yet. What can I do?

  Read the article

• Limitation of high level languages? [closed]

  - by user1705796

  My question may look bit philosophical and nonsense! But I need to know kind of instructions those are not well suitable in high level languages even in c? Or rarely use in the development of software? Like read/write content of CPU registers may useful in debugging programs. And access to cache memory required when developing OS (maybe I am wrong at this point). Is this kind of instruction available languages like Java, Python, C? I also have a second question: And Why all high level languages not having same uniform syntax; at-least same standard library interface name? In python there is and. Or operator is almost same as && and ||. I think Python is developed after C but space indentation is compulsory in Python. Why Python does not use brackets {}. I already know this question going to be highly down-voted.

  Read the article

< Previous Page | 146 147 148 149 150 151 152 153 154 155 156 157  | Next Page >