Search Results

Search found 31421 results on 1257 pages for 'software performance'.

Page 279/1257 | < Previous Page | 275 276 277 278 279 280 281 282 283 284 285 286  | Next Page >

  • Mysql regexp performance question

    - by Tim
    Rumour has it that this; SELECT * FROM lineage_string where lineage like '%179%' and lineage regexp '(^|/)179(/|$)' Would be faster than this; SELECT * FROM lineage_string where lineage regexp '(^|/)179(/|$)' Can anyone confirm ? Or know a decent way to test the speed of such queries. Thanks

    Read the article

  • Efficient file buffering & scanning methods for large files in python

    - by eblume
    The description of the problem I am having is a bit complicated, and I will err on the side of providing more complete information. For the impatient, here is the briefest way I can summarize it: What is the fastest (least execution time) way to split a text file in to ALL (overlapping) substrings of size N (bound N, eg 36) while throwing out newline characters. I am writing a module which parses files in the FASTA ascii-based genome format. These files comprise what is known as the 'hg18' human reference genome, which you can download from the UCSC genome browser (go slugs!) if you like. As you will notice, the genome files are composed of chr[1..22].fa and chr[XY].fa, as well as a set of other small files which are not used in this module. Several modules already exist for parsing FASTA files, such as BioPython's SeqIO. (Sorry, I'd post a link, but I don't have the points to do so yet.) Unfortunately, every module I've been able to find doesn't do the specific operation I am trying to do. My module needs to split the genome data ('CAGTACGTCAGACTATACGGAGCTA' could be a line, for instance) in to every single overlapping N-length substring. Let me give an example using a very small file (the actual chromosome files are between 355 and 20 million characters long) and N=8 import cStringIO example_file = cStringIO.StringIO("""\ header CAGTcag TFgcACF """) for read in parse(example_file): ... print read ... CAGTCAGTF AGTCAGTFG GTCAGTFGC TCAGTFGCA CAGTFGCAC AGTFGCACF The function that I found had the absolute best performance from the methods I could think of is this: def parse(file): size = 8 # of course in my code this is a function argument file.readline() # skip past the header buffer = '' for line in file: buffer += line.rstrip().upper() while len(buffer) = size: yield buffer[:size] buffer = buffer[1:] This works, but unfortunately it still takes about 1.5 hours (see note below) to parse the human genome this way. Perhaps this is the very best I am going to see with this method (a complete code refactor might be in order, but I'd like to avoid it as this approach has some very specific advantages in other areas of the code), but I thought I would turn this over to the community. Thanks! Note, this time includes a lot of extra calculation, such as computing the opposing strand read and doing hashtable lookups on a hash of approximately 5G in size. Post-answer conclusion: It turns out that using fileobj.read() and then manipulating the resulting string (string.replace(), etc.) took relatively little time and memory compared to the remainder of the program, and so I used that approach. Thanks everyone!

    Read the article

  • Mysql select - improve performances

    - by realshadow
    Hey, I am working on an e-shop which sells products only via loans. I display 10 products per page in any category, each product has 3 different price tags - 3 different loan types. Everything went pretty well during testing time, query execution time was perfect, but today when transfered the changes to the production server, the site "collapsed" in about 2 minutes. The query that is used to select loan types sometimes hangs for ~10 seconds and it happens frequently and thus it cant keep up and its hella slow. The table that is used to store the data has approximately 2 milion records and each select looks like this: SELECT * FROM products_loans WHERE KOD IN("X17/Q30-10", "X17/12", "X17/5-24") AND 369.27 BETWEEN CENA_OD AND CENA_DO; 3 loan types and the price that needs to be in range between CENA_OD and CENA_DO, thus 3 rows are returned. But since I need to display 10 products per page, I need to run it trough a modified select using OR, since I didnt find any other solution to this. I have asked about it here, but got no answer. As mentioned in the referencing post, this has to be done separately since there is no column that could be used in a join (except of course price and code, but that ended very, very badly). Here is the show create table, kod and CENA_OD/CENA_DO very indexed via INDEX. CREATE TABLE `products_loans` ( `KOEF_ID` bigint(20) NOT NULL, `KOD` varchar(30) NOT NULL, `AKONTACIA` int(11) NOT NULL, `POCET_SPLATOK` int(11) NOT NULL, `koeficient` decimal(10,2) NOT NULL default '0.00', `CENA_OD` decimal(10,2) default NULL, `CENA_DO` decimal(10,2) default NULL, `PREDAJNA_CENA` decimal(10,2) default NULL, `AKONTACIA_SUMA` decimal(10,2) default NULL, `TYP_VYHODY` varchar(4) default NULL, `stage` smallint(6) NOT NULL default '1', PRIMARY KEY (`KOEF_ID`), KEY `CENA_OD` (`CENA_OD`), KEY `CENA_DO` (`CENA_DO`), KEY `KOD` (`KOD`), KEY `stage` (`stage`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 And also selecting all loan types and later filtering them trough php doesnt work good, since each type has over 50k records and the select takes too much time as well... Any ides about improving the speed are appreciated. Edit: Here is the explain +----+-------------+----------------+-------+---------------------+------+---------+------+--------+-------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+----------------+-------+---------------------+------+---------+------+--------+-------------+ | 1 | SIMPLE | products_loans | range | CENA_OD,CENA_DO,KOD | KOD | 92 | NULL | 190158 | Using where | +----+-------------+----------------+-------+---------------------+------+---------+------+--------+-------------+ I have tried the combined index and it improved the performance on the test server from 0.44 sec to 0.06 sec, I cant access the production server from home though, so I will have to try it tomorrow.

    Read the article

  • Improve performance writing 10 million records to text file using windows service

    - by user1039583
    I'm fetching more than 10 millions of records from database and writing to a text file. It takes hours of time to complete this operation. Is there any option to use TPL features here? It would be great if someone could get me started implementing this with the TPL. using (FileStream fStream = new FileStream("d:\\file.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite)) { BufferedStream bStream = new BufferedStream(fStream); TextWriter writer = new StreamWriter(bStream); for (int i = 0; i < 100000000; i++) { writer.WriteLine(i); } bStream.Flush(); writer.Flush(); // empty buffer; fStream.Flush(); }

    Read the article

  • Stopping cookies being set from a domain (aka "cookieless domain") to increase site performance

    - by Django Reinhardt
    I was reading in Google's documentation about improving site speed. One of their recommendations is serving static content (images, css, js, etc.) from a "cookieless domain": Static content, such as images, JS and CSS files, don't need to be accompanied by cookies, as there is no user interaction with these resources. You can decrease request latency by serving static resources from a domain that doesn't serve cookies. Google then says that the best way to do this is to buy a new domain and set it to point to your current one: To reserve a cookieless domain for serving static content, register a new domain name and configure your DNS database with a CNAME record that points the new domain to your existing domain A record. Configure your web server to serve static resources from the new domain, and do not allow any cookies to be set anywhere on this domain. In your web pages, reference the domain name in the URLs for the static resources. This is pretty straight forward stuff, except for the bit where it says to "configure your web server to serve static resources from the new domain, and do not allow any cookies to be set anywhere on this domain". From what I've read, there's no setting in IIS that allows you to say "serve static resources", so how do I prevent ASP.NET from setting cookies on this new domain? At present, even if I'm just requesting a .jpg from the new domain, it sets a cookie on my browser, even though our application's cookies are set to our old domain. For example, ASP.NET sets an ".ASPXANONYMOUS" cookie that (as far as I'm aware) we're not telling it to do. Apologies if this is a real newb question, I'm new at this! Thanks.

    Read the article

  • pyInotify performance

    - by tranimatronic
    I have a very large directory tree I am wanting pyInotify to watch. Is it better to have pyInotify watch the entire tree or is it better to have a number of watches reporting changes to specific files ? Thanks

    Read the article

  • Best Tools for Software Maintenance Engineering

    - by Pev
    Yes, the dreaded 'M' word. You've got a workstation, source control and half a million lines of source code that you didn't write. The documentation was out of date the moment that it was approved and published. The original developers are LTAO, at the next project/startup/loony bin and not answering email. What are you going to do? {favourite editor} and Grep will get you started on your spelunking through the gnarling guts of the code base but what other tools should be in the maintenance engineers toolbox? To start the ball-rolling; I don't think I could live without source-insight for C/C++ spelunking. (DISCLAIMER: I don't work for 'em).

    Read the article

  • Performance implications of finalizers on JVM

    - by Alexey Romanov
    According to this post, in .Net, Finalizers are actually even worse than that. Besides that they run late (which is indeed a serious problem for many kinds of resources), they are also less powerful because they can only perform a subset of the operations allowed in a destructor (e.g., a finalizer cannot reliably use other objects, whereas a destructor can), and even when writing in that subset finalizers are extremely difficult to write correctly. And collecting finalizable objects is expensive: Each finalizable object, and the potentially huge graph of objects reachable from it, is promoted to the next GC generation, which makes it more expensive to collect by some large multiple. Does this also apply to JVMs in general and to HotSpot in particular?

    Read the article

  • Version Control software (client server model)

    - by Amira Elsayed
    Hi All, I'm using windows 7 , and all what i want is to install Version Control server on one machine and let other developers to connect to it using the machine IP address and chekout, update and commit files I have tried VisualSVN and it works well for me , I also have tried to install Apache Server and try to configure it to run with subversion but I failed to do so , so if any one can help me I will appreciated Thanks in Advance Edit what I want if any one can suggest an alternative like VisualSVN that let me compare and choose from different options Thanks in Advance

    Read the article

  • How to test my GAE site for performance

    - by Sergey Basharov
    I am building a GAE site that uses AJAX/JSON for almost all its tasks including building the UI elements, all interactions and client-server requests. What is a good way to test it for highloads so that I could have some statistics about how much resources 1000 average users per some period of time would take. I think I can create some Python functions for this purpose. What can you advise? Thanks.

    Read the article

  • LINQ Joins - Performance

    - by Meiscooldude
    I am curious on how exactly LINQ (not LINQ to SQL) is performing is joins behind the scenes in relation to how Sql Server performs joins. Sql Server before executing a query, generates an Execution Plan. The Execution Plan is basically an Expression Tree on what it believes is the best way to execute the query. Each node provides information on whether to do a Sort, Scan, Select, Join, ect. On a 'Join' node in our execution plan, we can see three possible algorithms; Hash Join, Merge Join, and Nested Loops Join. Sql Server will choose which algorithm to for each Join operation based on expected number of rows in Inner and Outer tables, what type of join we are doing (some algorithms don't support all types of joins), whether we need data ordered, and probably many other factors. Join Algorithms: Nested Loop Join: Best for small inputs, can be optimized with ordered inner table. Merge Join: Best for medium to large inputs sorted inputs, or an output that needs to be ordered. Hash Join: Best for medium to large inputs, can be parallelized to scale linearly. LINQ Query: DataTable firstTable, secondTable; ... var rows = from firstRow in firstTable.AsEnumerable () join secondRow in secondTable.AsEnumerable () on firstRow.Field<object> (randomObject.Property) equals secondRow.Field<object> (randomObject.Property) select new {firstRow, secondRow}; SQL Query: SELECT * FROM firstTable fT INNER JOIN secondTable sT ON fT.Property = sT.Property Sql Server might use a Nested Loop Join if it knows there are a small number of rows from each table, a merge join if it knows one of the tables has an index, and Hash join if it knows there are a lot of rows on either table and neither has an index. Does Linq choose its algorithm for joins? or does it always use one?

    Read the article

  • Performance improvement of client server system

    - by Tanuj
    I have a legacy client server system where the server maintains a record of some data stored in a sqlite database. The data is related to monitoring access patterns of files stored on the server. The client application is basically a remote viewer of the data. When the client is launched, it connects to the server and gets the data from the server to display in a grid view. The data gets updated in real time on the server and the view in the client automatically gets refreshed. There are two problems with the current implementation: When the database gets too big, it takes a lot of time to load the client. What are the best ways to deal with this. One option is to maintain a cache at the client side. How to best implement a cache ? How can the server maintain a diff so that it only sends the diff during the refresh cycle. There can be multiple clients and each client needs to display the latest data available on the server. The server is a windows service daemon. Both the client and the server are implemented in C#

    Read the article

  • Software Company Library

    - by dbemerlin
    Hi. A few days ago i had the idea to create a company library since my company has no training and many developers still develop as they did when they learned it 5 years ago. My hope is that they can lend books, read them and hopefully learn something from them (for example: object oriented programming or unit testing, which noone here knows how to use). After asking around most agreed that it was a good idea, so i brought my books, made a simple printed sheet with "Book A belongs to B" and "Developer A took the Book on dd.mm.yyyy" to get it started. Now i want to get some ideas for Books that i could add to the shelf (sadly from my own money since 100€/month for training is too much money for this multi-million euro company). We develop mostly PHP & MySQL so books specific to this topic would be preferred but i think if people learn other languages they might get ideas on how to develop better with the current language so other books are ok, too. Which books would you recommend? PS: Personally i'd like to add some Project Management books, too, as it's a topic i'm interested in, eventhough i'm just a junior developer (We've got Peopleware already, great book btw).

    Read the article

  • How large is the performance loss for a 64-bit VirtualBox guest running on a 32-bit host?

    - by IllvilJa
    I have a 64-bit Virtualbox guest running Gentoo Linux (amd64) and it is currently hosted on a 32-bit Gentoo laptop. I've noticed that the performance of the VM is very slow compared to the performance of the 32-bit host itself. Also when I compare with another 32-bit Linux VM running on the same host, performance is significantly less on the 64-bit VM. I know that running a 64-bit VM on a 32-bit host does incur some performance penalties for the VM, but does anyone have any deeper knowledge of how large a penalty one might expect in this scenario, roughly speaking? Is a 10% slowdown something to expect, or should it be a slowdown in the 90% range (running at 1/10 the normal speed)? Or to phrase it in another way: would it be reasonable to expect that the performance improvement for the 64-bit VM increases so much that it is worth reinstalling the host machine to run 64-bit Gentoo instead? I'm currently seriously considering that upgrade, but am curious about other peoples experience of the current scenario. I am aware that the host OS will require more RAM when running in 64-bit, but that's OK for me. Also, I do know that one usually don't run a 64-bit VM on a 32-bit server (I'm surprised I even got the VM started in the first place) but things turned out that way when I tried to future proof the VM I was setting up and decided to make it 64-bit anyway.

    Read the article

  • SEO for Computer Software Engineering Topics

    - by Michael Aaron Safyan
    I'm currently trying to SEO my development and coding search custom search engine as well my website that has a variety of coding and development resources. I would like to increase the number of links to my website, but I don't want to simply generate spam. What are some places that I should submit my website, where the content would be considered relevant rather than just spam? Thanks.

    Read the article

  • How do you keep track of what the industry is up to?

    - by BlairHippo
    A discussion elsewhere made me realize that I don't do a particularly good job of following the software industry. My exposure to new trends or technologies is haphazard at best, often limited to a "Hey, that sounds interesting" when I see people discussing something I'm not familiar with on SO. To abuse a metaphor, I'm quite familiar with the tree where I work, but I know too bloody little about the rest of the forest. How do other folks keep abreast of what's going on in the software industry? Are there any sites/blogs/podcasts/whatever that you find particularly valuable for keeping you informed of potentially useful new technologies or industry-wide trends? (My apologies in advance if this is a duplicate; this feels like something that ought to have been asked before, but alas, my search-fu has failed me.)

    Read the article

  • ESXi with software iSCSI

    - by jharley
    Has anyone had any luck using the swiSCSI driver on ESXi? Following the instructions from VMWare.com I get to the point where I have the iSCSI HBA showing up but no LUNs/targets are showing up. The iSCSI target is running on Solaris 10 update 5 and works with other initiators fine. The ESXi initiator (from the logs) sees the targets but just logs in and out of them every 2 - 5 seconds. We're using unauthenticated discovery, and over and over in /var/log/messages I see: iSCSI: bus 0 target 0 trying to establish session 0xb203f90 to portal 0, address 10.1.100.9 port 3260 group 1 iSCSI: bus 0 target 0 establish session 0xb203f90 #4848 to port 0, address 10.1.100.9 port 3260 group 1, alias data/ESXi iSCSI: session 0xb203f90 dropping after receiving unexpected opcode 0x60 iSCSI: session 0xb203f90 to data/ESXi dropped iSCSI: session 0xb203f90 to data/ESXi waiting 2 seconds before next login attempt The only other thing that seems out of wack is that my 'Recent Tasks' pane keeps filling with 'Browse Diagnostic Manager' events and /var/log/vmware/hostd.log is filled with messages like this up to two times per second: [2008-09-19 16:05:57.901 'TaskManager' 196621 info] Task Created: haTask-ha-host-vim.DiagnosticManager.browser-776 [2008-09-19 16:05:57.094 'TaskManager' 196621 info] Task Completed: haTask-ha-host-vim.DiagnosticManager.browser-766 Any help would be appreciated.

    Read the article

  • Performance problems when loading local JSON via <script> elements in IE8

    - by Jens Bannmann
    I have a web page with some JS scripts that needs to work locally, e.g. from hard disk or a CD-ROM. The scripts load JSON data from files by inserting <script> tags. This worked fine in IE6, but now in IE8 it takes an enormous amount of time: it went from "instantly" to 3-10 seconds. The main data file is 45KB large. How can I solve this? I would switch from <script> tags to another method of loading JSON (ideally involving the new native JSON parser), but it seems locally loaded content cannot access the XMLHttpRequest object. Any ideas?

    Read the article

< Previous Page | 275 276 277 278 279 280 281 282 283 284 285 286  | Next Page >