Search Results

Search found 14861 results on 595 pages for 'high speed computing'.

Page 483/595 | < Previous Page | 479 480 481 482 483 484 485 486 487 488 489 490  | Next Page >

  • Error Converting PIL B&W images to Numpy Arrays

    - by Elliot
    I am getting weird errors when I try to convert a black and white PIL image to a numpy array. An example of the code I am working with is below. if image.mode != '1': image = image.convert('1') #convert to B&W data = np.array(image) #convert data to a numpy array n_lines = data.shape[0] #number of raster passes line_range = range(data.shape[1]) for l in range(n_lines): # process one horizontal line of the image line = data[l] for n in line_range: if line[n] == 1: write_line_to(xl, z+scale*n, speed) #conversion to other program code elif line[n] == 0: run_to(xl, z+scale*n) #conversion to other program code I have tried this using both array and asarray for the conversion, and gotten different errors. If I use array, then the data I get out is nothing like what I put in. It looks like several very shrunken partial images side by side, with the remainder of the image space filled in in black. If I use asarray, then the entirety of python crashes during the raster step (on a random line). If I work with a greyscale image ('L'), then neither of these errors occurs for either array or asarray. Does anyone know what I am doing wrong? Is there something odd about the way PIL encodes B&W images, or something special I need to pass numpy to make it convert properly?

    Read the article

  • Speedup writing C programs using a subset of the Python syntax

    - by psihodelia
    I am constantly trying to optimize my time. Writing a C code takes a lot of time and requires much more keyboard touches than say writing a Python program. However, in order to speed up the time required to create a C program, one can automatize many things. I'd like to write my programs using smth. like Python but with C semantics. It means, all keywords are C keywords, but syntax is optimized. For example, this C code: #include "dsplib.h" #include "coeffs.h" #define MODULENAME "dsplib" #define NUM_SAMPLES 320 typedef float t_Vec; typedef struct s_Inter { char *pc_Name; struct s_Inter *px_Next; }t_Inter; typedef struct s_DspLibControl { t_Vec f_Y; }t_DspLibControl; void v_DspLibName(void) { printf("Module: %s", MODULENAME); printf("\n"); } int v_DspLibInitInterControl(t_DspLibControl *px_Con) { int y; px_Con->f_Y = 0.0; for(int i=0;i<10;i++) { y += i * i; } return y; } in optimized pythonized version can look like: include dsplib, coeffs define MODULENAME="dsplib", NUM_SAMPLES=320 typedef float t_Vec typedef struct s_Inter: char *pc_Name struct s_Inter *px_Next t_Inter typedef struct s_DspLibControl: t_Vec f_Y t_DspLibControl v_DspLibName(): printf("Module: %s", MODULENAME); printf("\n") int v_DspLibInitInterControl(t_DspLibControl *px_Con): int y px_Con->f_Y = 0.0 for int i=0;i<10;i++: y += i * i return y My question is: Do you know any VIM script, which allows to translate an original pythonized C code into a standard C code? For example, one is writing a C code but uses pythonized syntax, once she decides to translate pythonized blocks into standard C, she selects such blocks and press some key. And she doesn't save such pythonized code of course, VIM translates it into standard C.

    Read the article

  • Hiring my first employee

    - by Ady
    A few years ago I moved to a new job having been programming for 2 years using C#, however this new company was mainly using VB6. I made the case for .NET and won, but one of the consessions I had to make was to use VB.NET and not C# (understandable as most of the other developers were already using VB). Three years later it was time to move on, but when applying for jobs I couldn't get past the recruitment agents. I realised that when they were looking at the basic requirements (5 years experience) that they could not add 2 and 3 together to make 5. They were looking for 5 years in VB or C# not across both. Frustrated I decided to combine my skills with a designer friend and start my own company. After two years of hard graft we are now looking for our first employee (a programmer), and this question has hit me again, but now I see the employers perspective. Why take the risk of someone getting up to speed when you have thousands of applicants to choose from. So my question is this, if I define the requirements to be too narrow, I could miss the really great candidates. But if they are too broad it's going to take ages to go through them all. This will be our first 'employee' so the choice needs to be good, I can't afford to make a mistake and employ someone naff. Another option would be to choose a bright university graduate, and train them up (less of a risk because we can pay them less). What have others done in this situation, and what would you recommend I do?

    Read the article

  • RSA_sign and RSACryptoProvider.VerifySignature

    - by Miky D
    I'm trying to get up to speed on how to get some code that uses OpenSSL for cryptography, to play nice with another program that I'm writing in C#, using the Microsoft cryptography providers available in .NET. More to the point, I'm trying to have the C# program verify an RSA message signature generated by the OpenSSL code. The code that generates the signature looks something like this: // Code in C, using the OpenSSL RSA implementation char msgToSign[] = "Hello World"; // the message to be signed char signature[RSA_size(rsa)]; // buffer that will hold signature int slen = 0; // will contain signature size // rsa is an OpenSSL RSA context, that's loaded with the public/private key pair memset(signature, 0, sizeof(signature)); RSA_sign(NID_sha1 , (unsigned char*)msgToSign , strlen(msgToSign) , signature , &slen , rsa); // now signature contains the message signature // and can be verified using the RSA_verify counterpart // .. I would like to verify the signature in C# In C#, I would do the following: import the other side's public key into an RSACryptoServiceProvider object receive the message and it's signature try to verify the signature I've got the first two parts working (I've verified that the public key is loading properly because I managed to send an RSA encrypted text from the C# code to the OpenSSL code in C and successfully have it decrypted) In order to verify the signature in C#, I've tried using the: VerifySignature method of the RSACryptoServiceProvider but that didn't work. And digging around the internet I was only able to find some vague information pointing out that .NET uses a different method for generating the signature than OpenSSL does. So, does anybody know how to accomplish this?

    Read the article

  • Best strategy for synching data in iPhone app

    - by iamj4de
    I am working on a regular iPhone app which pulls data from a server (XML, JSON, etc...), and I'm wondering what is the best way to implement synching data. Criteria are speed (less network data exchange), robustness (data recovery in case update fails), offline access and flexibility (adaptable when the structure of the database changes slightly, like a new column). I know it varies from app to app, but can you guys share some of your strategy/experience? For me, I'm thinking of something like this: 1) Store Last Modified Date in iPhone 2) Upon launching, send a message like getNewData.php?lastModifiedDate=... 3) Server will process and send back only modified data from last time. 4) This data is formatted as so: <+><data id="..."></data></+> // add this to SQLite/CoreData <-><data id="..."></data></-> // remove this <%><data id="..."><attribute>newValue</attribute></data></%> // new modified value I don't want to make <+, <-, <%... for each attribute as well, because it would be too complicated, so probably when receive a <% field, I would just remove the data with the specified id and then add it again (assuming id here is not some automatically auto-incremented field). 5) Once everything is downloaded and updated, I will update the Last Modified Date field. The main problem with this strategy is: If the network goes down when I am updating something = the Last Modified Date is not yet updated = next time I relaunch the app, I will have to go through the same thing again. Not to mention potential inconsistent data. If I use a temporary table for update and make the whole thing atomic, it would work, but then again, if the update is too long (lots of data change), the user has to wait a long time until new data is available. Should I use Last-Modified-Date for each of the data field and update data gradually?

    Read the article

  • Handling learning curve for new developers

    - by pete the pagan-gerbil
    Our company likes to hire new developers, with no experience. We have a core set of skills that we try to get them up to speed with, like ASP.NET and WinForms - to teach basic programming, the .NET languages, and the things they'll need to maintain and write. We also try and mentor them through early projects, so they can learn from someone more experienced. Recently, we've been seeing the benefits of new frameworks like MVC and ideas like Unit Testing and TDD (by extension, dependancy injection and IoC), and we'd like to start using these in the team. However, this increases the time that a junior would have before they can get started on a new project - because doing something like unit tests wrong could cause major headaches months or years later in maintenance, especially if we believe unit tests to be comprehensive. How do you handle the huge amount of things that a junior will need to take on, acknowledging that the business wants them working independantly as soon as possible? Is it acceptable to tell them not to unit test till a while after they are independant (and give them small, simpler projects in the meantime) before taking them to 'level 2' of the core skills?

    Read the article

  • C# vs C - Big performance difference

    - by John
    I'm finding massive performance differences between similar code in C anc C#. The C code is: #include <stdio.h> #include <time.h> #include <math.h> main() { int i; double root; clock_t start = clock(); for (i = 0 ; i <= 100000000; i++){ root = sqrt(i); } printf("Time elapsed: %f\n", ((double)clock() - start) / CLOCKS_PER_SEC); } And the C# (console app) is: using System; using System.Collections.Generic; using System.Text; namespace ConsoleApplication2 { class Program { static void Main(string[] args) { DateTime startTime = DateTime.Now; double root; for (int i = 0; i <= 100000000; i++) { root = Math.Sqrt(i); } TimeSpan runTime = DateTime.Now - startTime; Console.WriteLine("Time elapsed: " + Convert.ToString(runTime.TotalMilliseconds/1000)); } } } With the above code, the C# completes in 0.328125 seconds (release version) and the C takes 11.14 seconds to run. The c is being compiled to a windows executable using mingw. I've always been under the assumption that C/C++ were faster or at least comparable to C#.net. What exactly is causing the C to run over 30 times slower? EDIT: It does appear that the C# optimizer was removing the root as it wasn't being used. I changed the root assignment to root += and printed out the total at the end. I've also compiled the C using cl.exe with the /O2 flag set for max speed. The results are now: 3.75 seconds for the C 2.61 seconds for the C# The C is still taking longer, but this is acceptable

    Read the article

  • What is SSIS order of data transformation component method calls

    - by Ron Ruble
    I am working on a custom data transformation component. I'm using NUnit and NMock2 to test as I code. Testing and getting the custom UI and other features right is a huge pain, in part because I can't find any documentation about the order in which SSIS invokes methods on the component at design time as well as runtime. I can correct the issues readily enough, but it's tedious and time consuming to unregister the old version, register the new version, fire up the test ssis package, try to display the UI, get an obscure error message, backtrace it, modify the component and continue. One of the big issues involves the UI component needing access to the componentmetadata and buffermanager properties of the component at design time, and what I need to provide for to support properties that won't be initialized until after the user enters them in the UI. I can work through it; but if someone knows of some docs or tips that would speed me up, I'd greatly appreciate it. The samples I've found havn't been much use; they seem to be directed to showing off cool stuff (Twitter, weather.com) rather than actual work. Thanks in advance.

    Read the article

  • What do you use to play sound in iPhone games?

    - by zoul
    Hello! I have a performance-intensive iPhone game I would like to add sounds to. There seem to be about three main choices: (1) AVAudioPlayer, (2) Audio Queues and (3) OpenAL. I’d hate to write pages of low-level code just to play a sample, so that I would like to use AVAudioPlayer. The problem is that it seems to kill the performace – I’ve done a simple measuring using CFAbsoluteTimeGetCurrent and the play message seems to take somewhere from 9 to 30 ms to finish. That’s quite miserable, considering that 25 ms == 40 fps. Of course there is the prepareToPlay method that should speed things up. That’s why I wrote a simple class that keeps several AVAudioPlayers at its disposal, prepares them beforehand and then plays the sample using the prepared player. No cigar, still it takes the ~20 ms I mentioned above. Such performance is unusable for games, so what do you use to play sounds with a decent performance on iPhone? Am I doing something wrong with the AVAudioPlayer? Do you play sounds with Audio Queues? (I’ve written something akin to AVAudioPlayer before 2.2 came out and I would love to spare that experience.) Do you use OpenAL? If yes, is there a simple way to play sounds with OpenAL, or do you have to write pages of code? Update: Yes, playing sounds with OpenAL is fairly simple.

    Read the article

  • Temporary storage for keeping data between program iterations?

    - by mr.b
    I am working on an application that works like this: It fetches data from many sources, resulting in pool of about 500,000-1,500,000 records (depends on time/day) Data is parsed Part of data is processed in a way to compare it to pre-existing data (read from database), calculations are made, and stored in database. Resulting dataset that has to be stored in database is, however, much smaller in size (compared to original data set), and ranges from 5,000-50,000 records. This process almost always updates existing data, perhaps adds few more records. Then, data from step 2 should be kept somehow, somewhere, so that next time data is fetched, there is a data set which can be used to perform calculations, without touching pre-existing data in database. I should point out that this data can be lost, it's not irreplaceable (key information can be read from database if needed), but it would speed up the process next time. Application components can (and will be) run off different computers (in the same network), so storage has to be reachable from multiple hosts. I have considered using memcached, but I'm not quite sure should I do so, because one record is usually no smaller than 200 bytes, and if I have 1,500,000 records, I guess that it would amount to over 300 MB of memcached cache... But that doesn't seem scalable to me - what if data was 5x that amount? If it were to consume 1-2 GB of cache only to keep data in between iterations (which could easily happen)? So, the question is: which temporary storage mechanism would be most suitable for this kind of processing? I haven't considered using mysql temporary tables, as I'm not sure if they can persist between sessions, and be used by other hosts in network... Any other suggestion? Something I should consider?

    Read the article

  • Python Turtle Graphics, how to plot functions over an interval?

    - by TheDragonAce
    I need to plot a function over a specified interval. The function is f1, which is shown below in the code, and the interval is [-7, -3]; [-1, 1]; [3, 7] with a step of .01. When I execute the program, nothing is drawn. Any ideas? import turtle from math import sqrt wn = turtle.Screen() wn.bgcolor("white") wn.title("Plotting") mypen = turtle.Turtle() mypen.shape("classic") mypen.color("black") mypen.speed(10) while True: try: def f1(x): return 2 * sqrt((-abs(abs(x)-1)) * abs(3 - abs(x))/((abs(x)-1)*(3-abs(x)))) * \ (1 + abs(abs(x)-3)/(abs(x)-3))*sqrt(1-(x/7)**2)+(5+0.97*(abs(x-0.5)+abs(x+0.5))-\ 3*(abs(x-0.75)+abs(x+0.75)))*(1+abs(1-abs(x))/(1-abs(x))) mypen.penup() step=.01 startf11=-7 stopf11=-3 startf12=-1 stopf12=1 startf13=3 stopf13=7 def f11 (startf11,stopf11,step): rc=[] y = f1(startf11) while y<=stopf11: rc.append(startf11) #y+=step mypen.setpos(f1(startf11)*25,y*25) mypen.dot() def f12 (startf12,stopf12,step): rc=[] y = f1(startf12) while y<=stopf12: rc.append(startf12) #y+=step mypen.setpos(f1(startf12)*25, y*25) mypen.dot() def f13 (startf13,stopf13,step): rc=[] y = f1(startf13) while y<=stopf13: rc.append(startf13) #y+=step mypen.setpos(f1(startf13)*25, y*25) mypen.dot() f11(startf11,stopf11,step) f12(startf12,stopf12,step) f13(startf13,stopf13,step) except ZeroDivisionError: continue

    Read the article

  • MS Query Analizer/Management Studio replacement?

    - by kprobst
    I've been using SQL Server since version 6.5 and I've always been a bit amazed at the fact that the tools seem to be targeted to DBAs rather than developers. I liked the simplicity and speed of the Query Analizer for example, but hated the built-in editor, which was really no better than a syntax coloring-capable Notepad. Now that we have Management Studio the management part seems a bit better but from a developer standpoint the tools is even worse. Visual Studio's excellent text editor... without a way to customize keyboard bindings!? Don't get me started on how unusable is the tree-based management hierarchy. Why can't I re-root the tree on a list of stored procs for example the way the Enterprise Manager used to allow? Now I have a treeview that needs to be scrolled horizontally, which makes it eminently useless. The SQL server support in Visual Studio is fantastic for working with stored procedures and functions, but it's terrible as a general ad hoc data query tool. I've tried various tools over the years but invariably they seem to focus on the management side and shortchange the developer in me. I just want something with basic admin capabilities, good keyboard support and requisite DDL functionality (ideally something like the Query Analyzer). At this point I'm seriously thinking of using vim+sqlcmd and a console... I'm that desperate :) Those of you who work day in and day out with SQL Server and Visual Studio... do you find the tools to be adequate? Have you ever wished they were better and if you have found something better, could you share please? Thanks!

    Read the article

  • Assembly unavailable after Web.config change

    - by tags2k
    I'm using a custom framework that uses reflection to do a GetTypeByName(string fullName) on the fully-qualified type name that it gets from the database, to create an instance of said type and add it to the page, resulting in a standard modular kind of thing. GetTypeByName is a utility function of mine that simply iterates through Thread.GetDomain().GetAssemblies(), then performs an assembly.GetType(fullName) to find the relevant type. Obviously this result gets cached for future reference and speed. However, I'm experiencing some issues whereby if the web.config gets updated (and, in some scarier instances if the application pool gets recycled) then it will lose all knowledge of certain assemblies, resulting in the inability to render an instance of the module type. Debugging shows that the missing assembly literally does not exist in the current thread assemblies list. To get around this I added a second check which is a bit dirty but recurses through the /bin/ directory's DLLs and checks that each one exists in the assemblies list. If it doesn't, it loads it using Assembly.Load and fixing the context issue thanks to 'Solving the Assembly Load Context Problem'. This would work, only it seems that (and I'm aware this shouldn't be possible) some projects still have access to the missing assembly, for example my actual web project rather than the framework itself - and it then complains that duplicate references have been added! Has anyone ever heard of anything like this, or have any ideas why an assembly would simply drop out of existence on a config change? Short of a solution, what is the most elegant workaround to get all the assemblies in the bin to reload? It needs to be all in one "hit" so that the site visitors don't see any difference other than a small delay, so an app_offline.htm file is out of the question. Programatically renaming a DLL in the bin and then naming it back does work, but requires "modify" permissions for the IIS user account, which is insane. Thanks for any pointers the community can gather!

    Read the article

  • Are there any configurable parameters to the gpsd?

    - by danatel
    I use the gpsd daemon with my application. Sometimes, the gpsd ceases to work with no apparent reason (clean sky). Even the gpsmon monitor shows no fix. Are there any parameters which must be set? Or is it a hardware problem? I am surprissed that many satellites are visible but the "Stat" bitmap does not contain the bit 7 - ephemeris data available. Should i somewhat pre-configure my position to allow for correct ephemeris data? Here is my gpsmon screen: 127.0.0.1:2947:/dev/ttyS3 SiRF binary> ^[[4~ -¦¦¦¦¦¦¦¦¦¦¦ X ¦¦¦¦¦¦ Y ¦¦¦¦¦¦ Z ¦¦¦¦¦¦¦¦¦¦ North ¦¦¦¦ East ¦¦¦¦¦ Alt ¦¦¦¦¦¦¦¦¦¬ -Pos: 3949260 1166016 4856299 m 49.89411° 16.44920° 1379 m - -Vel: 0.0 0.0 0.0 m/s 0.0 0.0 0.0 climb m/s- -Week+TOW:1578+224837.06 Day: 2 14:27:17.06 Heading: 0.0° 0.0 speed m/s- -Skew: -13.025817 TZ: -7200 HDOP: 0.0 M1:00 M2: 00 - -Fix: 0 = - L¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦ Packet type 2 (0x02) ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦- -¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¬-¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¬ -Ch PRN Az El Stat C/N ? A --Version: - - 0 2 243 19 003f 40.4 -L¦¦¦¦¦¦¦ Packet Type 6 (0x06) ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦- - 1 10 249 68 003f 43.0 --¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¬ - 2 13 90 30 003f 40.9 --SVs: 0 Drift: 96506 Bias: 135976716 - - 3 7 66 67 003f 39.8 --Estimated GPS Time: 224837059 - - 4 5 295 49 003d 39.7 -L¦¦¦¦¦¦¦ Packet type 7 (0x07) ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦- - 5 8 210 69 003f 41.0 --¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¬ - 6 23 96 5 002d 28.0 --Max: 167.570Lat: 132.129Time: 0.075 MS: 02 - - 7 6 43 3 002d 23.1 -L¦¦¦¦¦¦¦ Packet type 9 (0x09) ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦- - 8 28 163 16 003f 39.8 --¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¬ - 9 0 0 0 0000 0.0 --SVs: 11 = 8 10 7 5 13 2 28 23 3 6 4 - -10 3 55 4 002d 24.7 -L¦¦¦¦¦¦¦ Packet type 13 (0x0D) ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦- -11 0 0 0 0000 0.0 --¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¬ L¦¦¦ Packet Type 4 (0x04) ¦¦¦--DGPS source: 1 (SBAS) Corrections: 12 - L¦¦¦¦¦¦¦ Packet type 27 (0x1B) ¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦-q

    Read the article

  • Poor DB4O performance - What am I doing wrong?

    - by Jon
    I read Rob Conery's post about Object databases and thought I'd give it a go. I have a class lets say Book with 15 properties mostly string, some int, one Datetime and 2 decimals. I used Rob's code to open the database file etc to test on a Winforms project not a Web project. So I did the following: for(int i = 0; i < 1000000; i++) { var Book = new Book(); //Assign variables s.Save(Book); } s.CommitChanges(); This took at least a couple of minutes. I then tried to retrieve all data to test speed: var spp = s.All<Passports>(); This one line was quick, however, then doing: sppp.Count() as well as a seperate test doing a foreach loop and incrementing a counter took a couple of minutes. This seems quite slow to me. Am I doing something wrong or am I expecting too much?

    Read the article

  • How can I initialize a QTMovie object with certain attributes using writable data?

    - by c-had
    I'm trying to create an empty QTMovie object that I can add segments to, and then play. This is easy to do with something like: movie = [[QTMovie alloc] initToWritableData:[NSMutableData dataWithCapacity:1048576] error:&error]; I can then use -insertSegmentOfMovie to insert segments from other movies into this one so I can play it back. The problem is that I also need to set a certain attribute when creating the QTMovie object. In particular, I need to set the QTMovieRateChangesPreservePitchAttribute attribute, so that I can alter playback speed during playback without changing pitch. This attribute cannot be written after the movie is initialized. So, I can create the QTMovie object like this: movie = [[QTMovie alloc] initWithAttributes:[NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithBool:YES], QTMovieRateChangesPreservePitchAttribute, nil] error:&error]; Unfortunately, this is not editable. I've tried setting the QTMovieEditableAttribute as well on creation, but it does not help. I still get an exception when I try to insert anything into this movie. I presume this is because there is no writable file or data reference associated with the QTMovie. Any ideas on how to solve this?

    Read the article

  • Using Partitions for a large MySQL table

    - by user293594
    An update on my attempts to implement a 505,000,000-row table on MySQL on my MacBook Pro: Following the advice given, I have partitioned my table, tr: i UNSIGNED INT NOT NULL, j UNSIGNED INT NOT NULL, A FLOAT(12,8) NOT NULL, nu BIGINT NOT NULL, KEY (nu), key (A) with a range on nu. nu ought to be a real number, but because I only have 6-d.p. accuracy and the maximum value of nu is 30000. I multiplied it by 10^8 made it a BIGINT - I gather one can't use FLOAT or DOUBLE values to PARTITION a MySQL table. Anyway, I have 15 partitions (p0: nu<25,000,000,000, p1: nu<50,000,000,000, etc.). I was thinking that this should speed up a typical to SELECT: SELECT * FROM tr WHERE nu>95000000000 AND nu<100000000000 AND A.>1. to something of the order of the same query on a table consisting of only the data in the relevant partition (<30 secs). But it's taking 30mins+ to return rows for queries within a partition and double that if the query is for rows spanning two (contiguous) partitions. I realise I could just have 15 different tables, and query them separately, but is there a way to do this 'automatically' with partitions? Has anyone got any suggestions?

    Read the article

  • Save memory in Python. How to iterate over the lines and save them efficiently with a 2million line

    - by skyl
    I have a tab-separated data file with a little over 2 million lines and 19 columns. You can find it, in US.zip: http://download.geonames.org/export/dump/. I started to run the following but with for l in f.readlines(). I understand that just iterating over the file is supposed to be more efficient so I'm posting that below. Still, with this small optimization, I'm using 10% of my memory on the process and have only done about 3% of the records. It looks like, at this pace, it will run out of memory like it did before. Also, the function I have is very slow. Is there anything obvious I can do to speed it up? Would it help to del the objects with each pass of the for loop? def run(): from geonames.models import POI f = file('data/US.txt') for l in f: li = l.split('\t') try: p = POI() p.geonameid = li[0] p.name = li[1] p.asciiname = li[2] p.alternatenames = li[3] p.point = "POINT(%s %s)" % (li[5], li[4]) p.feature_class = li[6] p.feature_code = li[7] p.country_code = li[8] p.ccs2 = li[9] p.admin1_code = li[10] p.admin2_code = li[11] p.admin3_code = li[12] p.admin4_code = li[13] p.population = li[14] p.elevation = li[15] p.gtopo30 = li[16] p.timezone = li[17] p.modification_date = li[18] p.save() except IndexError: pass if __name__ == "__main__": run()

    Read the article

  • What would you write in a consitution (law book) for a programmers' country?

    - by Developer Art
    After we have got our great place to talk about professional matters and sozialize online - on SO, I believe the next logical step would be to found our own country! I invite you all to participate and bring together your available resources. We will buy an island, better even a group of island so that we could establish states like .NET territories, Java land, Linux republic etc. We will build a society of programmers (girls - we need you too). To the organizational side, we're going to need some constitution or a law book. I suggest we write it together. I make it a wiki as it should be a cooperative effort. I'll open the work. Section 1. Programmers' rights. Every citizen has a right to an Internet connection 24/7. Every citizen can freely choose the field of interest Section 2. Programmers' obligations. Every citizen must embrace the changing nature of the profession and constanly educate himself Section 3. Law enforcement. Code duplication when can be avoided is punished by limiting the bandwith speed to 64Kbit for a period of one week. Using ugly hacks instead of refactoring code is punished by cutting the Internet connection for a period of one month. Usage of technologies older than 5/10 years is punished by restricting the web access to the sites last updated 5/10 years ago for a period of one month. Please feel free to modify and extend the list. We'll need to have it ready before we proceed formally with the country foundation. A purchase fund will be established shortly. Everyone is invited to participate.

    Read the article

  • Combining cache methods - memcache/disk based

    - by Industrial
    Hi! Here's the deal. We would have taken the complete static html road to solve performance issues, but since the site will be partially dynamic, this won't work out for us. What we have thought of instead is using memcache + eAccelerator to speed up PHP and take care of caching for the most used data. Here's our two approaches that we have thought of right now: Using memcache on all<< major queries and leaving it alone to do what it does best. Usinc memcache for most commonly retrieved data, and combining with a standard harddrive-stored cache for further usage. The major advantage of only using memcache is of course the performance, but as users increases, the memory usage gets heavy. Combining the two sounds like a more natural approach to us, even though the theoretical compromize in performance. Memcached appears to have some replication features available as well, which may come handy when it's time to increase the nodes. What approach should we use? - Is it stupid to compromize and combine the two methods? Should we insted be focusing on utilizing memcache and instead focusing on upgrading the memory as the load increases with the number of users? Thanks a lot!

    Read the article

  • How can I superimpose modified loess lines on a ggplot2 qplot?

    - by briandk
    Background Right now, I'm creating a multiple-predictor linear model and generating diagnostic plots to assess regression assumptions. (It's for a multiple regression analysis stats class that I'm loving at the moment :-) My textbook (Cohen, Cohen, West, and Aiken 2003) recommends plotting each predictor against the residuals to make sure that: The residuals don't systematically covary with the predictor The residuals are homoscedastic with respect to each predictor in the model On point (2), my textbook has this to say: Some statistical packages allow the analyst to plot lowess fit lines at the mean of the residuals (0-line), 1 standard deviation above the mean, and 1 standard deviation below the mean of the residuals....In the present case {their example}, the two lines {mean + 1sd and mean - 1sd} remain roughly parallel to the lowess {0} line, consistent with the interpretation that the variance of the residuals does not change as a function of X. (p. 131) How can I modify loess lines? I know how to generate a scatterplot with a "0-line,": # First, I'll make a simple linear model and get its diagnostic stats library(ggplot2) data(cars) mod <- fortify(lm(speed ~ dist, data = cars)) attach(mod) str(mod) # Now I want to make sure the residuals are homoscedastic qplot (x = dist, y = .resid, data = mod) + geom_smooth(se = FALSE) # "se = FALSE" Removes the standard error bands But does anyone know how I can use ggplot2 and qplot to generate plots where the 0-line, "mean + 1sd" AND "mean - 1sd" lines would be superimposed? Is that a weird/complex question to be asking?

    Read the article

  • Is it wise to use temporary tables?

    - by Industrial
    Hi guys, We have a mySQL database table for products. We are utilizing a cache layer to reduce database load, but we think that it's a good idea to minimize the actual data needed to be stored in the cache layer to speed up the application further. All the products in the database, that is visible to visitors have a price attached to them: The prices are stored in a different table, called prices . There are multiple price categories depending on which discount level each visitor (customer) applies to. From time to time, there are campaigns which means that a special price for each product is available. The special prices are stored in a table called specials. Is it a bad to make a temp table that binds the tables together? It would only have the neccessary information and would ofcourse be cached. -------------|-------------|------------ | productId | hasPrice | hasSpecial -------------|-------------|------------ 1 | 1 | 0 2 | 1 | 1 By doing such, it would be super easy to know if the specific product really has a price, without having to iterate through the complete prices or specials table each time a product should be listed or presented. Are temp tables a common thing for web applications or is it just bad design?

    Read the article

  • Helping Rails Newbies identify version-specific information on web pages

    - by corprew
    I am trying to help some people getting started programming on rails identify which version that advice found on web pages corresponds to, and am seeking advice and/or guides on how to do it so they don't have to rely on me and/or waste time trying outdated advice. Narrative: I am helping some people get up to speed on rails development, and their stock response to running into problems is searching google for advice. They're using 2.3.5 and thinking of moving to 3. The problem they're running into is that there's a lot of advice out there specific to older rails versions (2.2 for example being popular) that isn't identified. I can usually figure out when the pages are old pretty easily, but they can't (yet.) It seems like random web page authors don't identify which version they're using when they're using the current version, and not all pages are dated. This seems to be a general problem that will get worse -- current unadorned advice is usually 2.3.5 and older unadorned advice is 2.2.x at this point, but people are moving / will be moving to version 3 over the next while and newbies will be stuck looking at a bunch of deprecated/incompatible 2.3.x advice without realizing which version it is. Any advice / pointers / telltales?

    Read the article

  • pass a number value to a Timer, XML and AS3

    - by VideoDnd
    I want to pass a number value to a Timer. How do I do this? My number and integer values for other variables work fine. Error I get null object reference and coercion of value, because I'm not passing to 'timer' properly. I don't want to say my variable's a number, I want to say it has a number value. Variable //what I have now var timer:Timer; timer = new Timer(100); Path myXML.COUNT.text(); XML <?xml version="1.0" encoding="utf-8"?> <SESSION> <TIMER TITLE="speed">100</TIMER> </SESSION> Parse and Load //LOAD XML var myXML:XML; var myLoader:URLLoader = new URLLoader(); myLoader.load(new URLRequest("time.xml")); myLoader.addEventListener(Event.COMPLETE, processXML); //PARSE XML function processXML(e:Event):void { myXML = new XML(e.target.data);

    Read the article

  • How do I use IPTC/EXIF metadata to categorise photos?

    - by kbro
    Many photo viewing and editing applications allow you to examine and change EXIF and IPTC data in JPEG and other image files. For example, I can see things like shutter speed, aperture and orientation in the picture files that come off my Canon A430. There are many, many name/value pairs in all this metadata. But... What do I do if I want to store some data that doesn't have a build-in field name. Let's say I'm photographing an athletics competition and I want to tag every photo with the competitor's bib number. Can I create a "bib_number" field and assign it a values of "0001", "5478", "8124" etc, and then search for all photos with bib_number="5478"? I've spent a few hours searching and the best I can come up with is to put this custom information in the "keywords" field but this isn't quite what I'm after. With this socution I'd have to craft a query like "keywords contains bib_number_5478" whereas what I want it "bib_number is 5478". So do the EXIF and/or IPTC standards allow addtional user-defined field names? Thanks Kev

    Read the article

< Previous Page | 479 480 481 482 483 484 485 486 487 488 489 490  | Next Page >