Search Results

Search found 10215 results on 409 pages for 'ram usage'.

Page 276/409 | < Previous Page | 272 273 274 275 276 277 278 279 280 281 282 283  | Next Page >

• Memory efficient import many data files into panda DataFrame in Python

  - by richardh

  I import into a panda DataFrame a directory of |-delimited.dat files. The following code works, but I eventually run out of RAM with a MemoryError:. import pandas as pd import glob temp = [] dataDir = 'C:/users/richard/research/data/edgar/masterfiles' for dataFile in glob.glob(dataDir + '/master_*.dat'): print dataFile temp.append(pd.read_table(dataFile, delimiter='|', header=0)) masterAll = pd.concat(temp) Is there a more memory efficient approach? Or should I go whole hog to a database? (I will move to a database eventually, but I am baby stepping my move to pandas.) Thanks! FWIW, here is the head of an example .dat file: cik|cname|ftype|date|fileloc 1000032|BINCH JAMES G|4|2011-03-08|edgar/data/1000032/0001181431-11-016512.txt 1000045|NICHOLAS FINANCIAL INC|10-Q|2011-02-11|edgar/data/1000045/0001193125-11-031933.txt 1000045|NICHOLAS FINANCIAL INC|8-K|2011-01-11|edgar/data/1000045/0001193125-11-005531.txt 1000045|NICHOLAS FINANCIAL INC|8-K|2011-01-27|edgar/data/1000045/0001193125-11-015631.txt 1000045|NICHOLAS FINANCIAL INC|SC 13G/A|2011-02-14|edgar/data/1000045/0000929638-11-00151.txt

  Read the article

• Unable to set enviornment variable in platform builder win CE 6.0

  - by mukesh

  Actually while building the win ce project getting the two errors -: Error 1 BUILD: [00:0000000015:ERRORE] C:\WINCE600\PLATFORM\ICOP_Vortex86_60CS\SRC\OAL\OALLIB\obj\x86\debug_objects.mac: create file failed. 2 Error 3 BLDDEMO: There were errors building mytest I think it's comes, due to unset of environmnet variable. facing problem to set environment variable IMGRAM128 in the project proerties, giving the error :- "The variable IMGRAM128 is associated with the 128 RAM catalog item.Would you like to set this variable by adding catalog item in OS design? The varible will not be added to the environmmet tab.............."

  Read the article

• pyinstaller: 2 instances of my cherrypy app exe get executed.

  - by d.c

  I have a cherrypy app that I've made an exe with pyinstaller. now when I run the exe it loads itself twice into memory. Watching the taskmanager shows the first instance load into about 1k, then a second later a second instance of hte exe loads into about 3k ram. If I close the bigger one both processes die. If I close hte smaller one only that one dies. Loading the exe with subprocess, if I try to proc.kill(), it only kills the small one leaving the other running in memory. Is this a sideeffect of using cherrypy and pyinstaller together?

  Read the article

• JBoss Seam - Jetty - Virtualhosting

  - by Walter White

  Hi all, I am trying to cutback on the memory usage of my server and would like to optimize the architecture. I currently deploy 2 separate web applications to Jetty 6.1.22 that correspond to different virtualhosts. They have pretty much the same application stack except one has fewer components and are styled differently (content, images, css, etc.). If I change my design pattern over to EJB / EAR + 2 WARS embedded, will that lower the memory consumption? Will that give me a single instance of JBoss Seam, Quartz, and all of my components? They must use a different datasource. Thanks, Walter

  Read the article

• Best CPUs for speeding up compiling times of C++ w/ DistGCC

  - by Jay

  I'm putting together a distributed build farm with DistGCC to speed up our teams compile times and just looking for thoughts on which processors to use in the hosts. Are we going to get a noticeable decrease in time using 8 cores vs. 4-hyperthreaded cores? Big difference in time between i7 and Xeon? etc, etc. Just need advice from people who've put together kick-a build clusters. We've got a majority of the normal things to speed up builds in place (pre-compiled headers, ccache, local gigabit connections between them, tons of ram, etc) so please just give advice on the best processor to use. And money is a factor, but anythings doable if the performance increase is noticeable. Thanks. Jay

  Read the article

• PHP/MySQL database connection priority?

  - by Josh

  Hello, I have a production database where usage statistics for a service we're working on. I use php to periodically roll up different resolutions (day, week, month, year) of interesting statistics in buckets dictated by the resolution. The php application I've written "completes" its data when its run, such that it will calculate all the rolled-up statistics for the resolutions and periods since it was last run. This is useful if we want to turn this off to debug database performance issues, because I can turn it back on and have it complete its data set. The problem I have, is the calculations are fairly intensive and drive the QPS of the production database server up. Is there a way to set a "priority" on a particular database connection so that it will only use "off-cycles" to do these calculations? Maybe a proper response would be to replicate the tables I'm working on into a different stats database, but, unfortunately I don't have the resources in place to attempt such a thing (yet). Thanks in advance for any help, Josh

  Read the article

• Grails / GORM, read-only cache and transient fields

  - by Stephen Swensen

  Suppose I have the following Domain object mapping to a legacy table, utilizing read-only second-level cache, and having a transient field: class DomainObject { static def transients = ['userId'] Long id Long userId static mapping = { cache usage: 'read-only' table 'SOME_TABLE' } } I have a problem, references to DomainObject instances seem to be shared due to the caching, and thus transient fields are writing over each other. For example, def r1 = DomainObject.get(1) r1.userId = 22 def r2 = DomainObject.get(1) r2.userId = 34 assert r1.userId == 34 That is, r1 and r2 are references to the same instance. This is undesirable, I would like to cache the table data without sharing references. Any ideas?

  Read the article

• A good file path builder library for C#?

  - by Igor Brejc

  System.IO.Path in .NET is notoriously clumsy to work with. In my various projects I keep encountering the same usage scenarios which require repetitive, verbose and thus error-prone code snippets that use Path.Combine, Path.GetFileName, Path.GetDirectoryName, String.Format, etc. Scenarios like: changing the extension for a given file name changing the directory path for a given file name building a file path using string formatting (like "Package{0}.zip") building a path without resorting to using hard-coded directory delimiters like \ (since they don't work on Linux on Mono) etc etc Before starting to write my own PathBuilder class or something similar: is there a good (and proven) open-source implementation of such a thing in C#?

  Read the article

• form update too expensive to be executed in Winform.Timer.Tick

  - by Abruzzo Forte e Gentile

  Hi all I have a WinForm drawing a chart from available data. I programmed it so that every 1 secong the Winform.Timer.Tick event calls a function that: will dequeue all data available will add new points on the chart Right now data to be plotted is really huge and it takes a lot of time to be executed so to update my form. Also Winform.Timer.Tick relies on WM_TIMER , so it executes in the same thread of the Form. Theses 2 things are making my form very UNresponsive. What can I do to solve this issue? I thought the following: moving away from usage of Winform.Timer and start using a System.Threading.Timer use the IsInvokeRequired pattern so I will rely on the .NET ThreadPool. Since I have lots of data, is this a good idea? I have fear that at some point also the ThreadPool will be too long or too big. Can you give me your suggestion about my issue? Thank you very much! AFG

  Read the article

• COMPLETE list of HTML tag attributes which have a URL value?

  - by system PAUSE

  Besides the following, are there any HTML tag attributes that have a URL as their value? href attribute on tags: <link>, <a>, <area> src attribute on tags: <img>, <iframe>, <frame>, <embed>, <script>, <input> action attribute on tags: <form> data attribute on tags: <object> Looking for tags in wide usage, including non-standard tags and old browsers as well as HTML 4.01, HTML 5, and XHTML. Yes this question is kinda lightweight, but I googled around for about 45 minutes and didn't find this data centralized anywhere, so I figure it might help some other developer to have it here. Plus I'm sure I'm missing something. Feel free to repeat/reorganize this list in your answer. Upvoting the most complete answers will probably be most helpful to others.

  Read the article

• Why can I access private/protected methods using Object#send in Ruby?

  - by smotchkkiss

  The class class A private def foo puts :foo end public def bar puts :bar end private def zim puts :zim end protected def dib puts :dib end end instance of A a = A.new test a.foo rescue puts :fail a.bar rescue puts :fail a.zim rescue puts :fail a.dib rescue puts :fail a.gaz rescue puts :fail test output fail bar fail fail fail .send test [:foo, :bar, :zim, :dib, :gaz].each { |m| a.send(m) rescue puts :fail } .send output foo bar zim dib fail The question The section labeled "Test Output" is the expected result. So why can I access private/protected method by simply Object#send? Perhaps more important: What is the difference between public/private/protected in Ruby? When to use each? Can someone provide real world examples for private and protected usage?

  Read the article

• How to Open a Remote Text File using Server.MapPath in ASP.Net?

  - by NJTechGuy

  This is what I have right now for a file residing on the same server and it works. Dim FILENAME as String = Server.MapPath("Output.txt") Dim objStreamWriter as StreamWriter objStreamWriter = File.CreateText(FILENAME) dr = myCommand.ExecuteReader() While dr.Read() objStreamWriter.WriteLine("{0}|{1}|{2:yyyy-MM-dd}|{3:yyyy-MM-dd}", dr(0), dr(1), dr(2), dr(3)) End While objStreamWriter.Close() I was planning on FTPing the file to another server once done but is there a way to do it on a remote server? i.e create a text file on a remote server and write to it? Did not find any instances of that usage. How to use ServerXMLHTTP to replace Server.MapPath in this instance? Thank you so much for your time.

  Read the article

• Partial sorting algorithm

  - by Enriquev

  Hello, Say I have 50 million features, each feature comes from disk. At the beggining of my program, I handle each feature and depending on some conditions, I apply some modifications to some. A this point in my program, I am reading a feature from disk, processing it, and writing it back, because well I don't have enough ram to open all 50 million features at once. Now say I want to sort these 50 million features, is there any optimal algorithm to do this as I can't load everyone at the same time? Like a partial sorting algorithm or something like that?

  Read the article

• C# SqlBulkCopy and Data Entities

  - by KP

  Guys, My current project consists of 3 standard layers: data, business, and presentation. I would like to use data entities for all my data access needs. Part of the functionality of the app will that it will need to copy all data within a flat file into a database. The file is not so big so I can use SqlBulkCopy. I have found several articles regarding the usage of SqlBulkCopy class in .NET. However, all the articles are using DataTables to move data back and forth. Is there a way to use data entities along with SqlBulkCopy or will I have to use DataTables?

  Read the article

• How does Qt implement signals and slots?

  - by anton

  Can someone explain to me the basic idea of Qt signals&slots mechanism IMPLEMENTATION? I want to know what all those Q_OBJECT macros do "in plain C++". This question is NOT about signals&slots usage. added: I know that Qt uses moc compiler to transform Qt-C++ in plain C++. But what does moc do? I tried to read "moc_filename.cpp" files but I have no idea what can something like this mean void *Widget::qt_metacast(const char *_clname) { if (!_clname) return 0; if (!strcmp(_clname, qt_meta_stringdata_Widget)) return static_cast<void*>(const_cast< Widget*>(this)); return QDialog::qt_metacast(_clname); } Thanks in Advance, anton

  Read the article

• CPU consumption of my process

  - by Abruzzo Forte e Gentile

  Hi all I would like to use Performance Monitor to check the CPU consumption of my process. Right now I am working on a MultiCore machine. If I have a look at my process in TASK MANAGER I see that my process consumes 20% of CPU. If I start performance monitor, I select Process--% Processor Time I see values peaking up and over 100%. Do you know why and how to get the real measure? I also looked at the CPU consumption for all of my 4 cores, but I don't know exactly how to attribute consumption to my process. If you can suggest a link or url about how to read CPU usage I would really appreciate! Thanks a lot! AFG

  Read the article

• Capturing SOAP requests to an ASP.NET ASMX web service

  - by pcampbell

  Consider the requirement to log incoming SOAP requests to an ASP.NET ASMX web service. The task is to capture the raw XML being sent to the web service. The incoming message needs to be logged for debug inspection. The application already has its own logging library in use, so the ideal usage would be something like this: //string or XML, it doesn't matter. string incomingSoapRequest = GetSoapRequest(); Logger.LogMessage(incomingSoapRequest); Are there any easy solutions to capture the raw XML of the incoming SOAP requests? Which events would you handle to get access to this object and the relevant properties?

  Read the article

• Pass Data to Data Template Selector

  - by Michael Sync

  How do you guys pass the data (parameter) to DataTemplate Selector? The only one that I can think of is to use an attached property in DataTemplate Selector? Example: public class DisableWeekendsSelection : DataTemplateSelector { [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2211:NonConstantFieldsShouldNotBeVisible", Justification = "DependencyProperty")] public static readonly DependencyProperty Parameter = DependencyProperty.RegisterAttached("Parameter", typeof(ObservableCollection<Date>), typeof(DisableWeekendsSelection), new FrameworkPropertyMetadata(null, FrameworkPropertyMetadataOptions.BindsTwoWayByDefault)); public static ObservableCollection<Date> GetParameter(DependencyObject dp) { return dp.GetValue(Parameter) as ObservableCollection<Date>; } public static void SetParameter(DependencyObject dp, ObservableCollection<Date> value) { dp.SetValue(Parameter, value); } public override DataTemplate SelectTemplate(object item, DependencyObject container) { The problem with this approach is that I'm not able to get the value of Parameter in SelectTemplate method. Any suggestion would be appreciated. Thanks in advance.

  Read the article

• How To Make Moving News Bar in C# Desktop Application without Timer

  - by Ehab Sutan

  Hello, I'm making a desktop application in C# which contains a moving News Bar labels. I'm using a timer to move these labels but the problem is that when i make the interval of this timer low (1-10 for example) the application takes very high percentage of CPU Usage, And when i make it higher(200 -500 ) the movement of the labels becomes intermittent or not smooth movement even that the user may not be able to read the news in Comfortable way. Question is : Is there better way to make news Bar moving in a good manner without consuming high resources. Thanks.

  Read the article

• May a NSManagedObjectContext re-fault objects automatically?

  - by frenetisch applaudierend

  I am trying to create an application which allows background threads to update core data objects while the user might be reading the same data. My approach to this would be to use multiple NSManagedObjectContexts and then before a background thread does a -save: operation, I fetch the object the user is currently working on and fire the fault for all its properties and relationships recursively. This way I have all objects the user could act with in my NSManagedObjectContext without seeing the already updated values. But this can only work if the NSManagedObjectContext cannot decide himself that e.g. memory usage is too high, and starts faulting objects which I do not explicitly reference (other than through the NSManagedObject relationship). So the question is, can the NSManagedObjectContext decide that an object needs to be re-faulted, without intervention from my side? Thanks for your effort, Markus

  Read the article

• How can I exceed the 60% Memory Limit of IIS7 in ASP.NET Caching application

  - by evilknot

  Pardon if this is more serverfault vs. stackoverflow. It seems to be on the border. We have an application that caches a large amount of product data for an e-commerce application using ASP.NET caching. This is a dictionary object with 65K elements, and our calculations put the object's size at ~10GB. Problem: The amount of memory the object consumes seems to be far in excess of our 10GB calculation. BIGGEST CONCERN: We can't seem to use over 60% of the 32GB in the server. What we've tried so far: In machine.config/system.web (sf doesn't allow the tags, pardon the formatting): processModel autoConfig="true" memoryLimit="80" In web.config/system.web/caching/cache (sf doesn't allow the tags, pardon the formatting): privateBytesLimit = "20000000000" (and 0, the default of course) percentagePhysicalMemoryUsedLimit = "90" Environment: Windows 2008R2 x64 32GB RAM IIS7 Nothing seems to allow us to exceed the 60% value. See screenshot of taskman. http://www.freeimagehosting.net/image.php?7a42144e03.jpg

  Read the article

• ASP.net - First build fails with 'Unknown server tag'

  - by Sophia

  I have multiple custom controls used on a ASPX (and C#) page registered from within the page rather than in Web.Config. On the first build (or rebuild), build fails with error messages for wherever I've used the custom controls. Subsequent builds are successful. The error message: Unknown server tag 'prefix:ExampleControl'. What might cause this, and how can I fix it? Register syntax: <%@ Register Src="ControlsFolder/ExampleControl.ascx" TagName="ExampleControl" TagPrefix="prefix" %> <!-- etc --> Usage syntax: <prefix:ExampleControl runat="server" ID="ExampleControl1" /> <!-- etc -->

  Read the article

• shell script output in html + email that html

  - by Kimi

  Using Solaris I have a monitoring script that uses other scripts as plugins. Theses pugins are also scripts which work in difffernt ways like: 1. Sending an alert while high memory uilization 2. High Cpu usage 3. Full disk Space 4. chekcking the core file dump Now all this is dispalyed on my terminal and I want to put them in a HTML file/format and send it as a body of the mail not as attachment. Thanks .

  Read the article

• Detect block size for quota in Linux

  - by Chen Levy

  The limit placed on disk quota in Linux is counted in blocks. However, I found no reliable way to determine the block size. Tutorials I found refer to block size as 512 bytes, and sometimes as 1024 bytes. I got confused reading a post on LinuxForum.org for what a block size really means. So I tried to find that meaning in the context of quota. I found a "Determine the block size on hard disk filesystem for disk quota" tip on NixCraft, that suggested the command: dumpe2fs /dev/sdXN | grep -i 'Block size' or blockdev --getbsz /dev/sdXN But on my system those commands returned 4096, and when I checked the real quota block size on the same system, I got a block size of 1024 bytes. Is there a scriptable way to determine the quota block size on a device, short of creating a known sized file, and checking it's quota usage?

  Read the article

• PHP Stop Word List

  - by Dom Hodgson

  I'm playing about with a stop words within my code I have an array full of words that I'd like to check, and an array of words I want to check against. At the moment I'm looping through the array one at at a time and removing the word if its in_array vs the stop word list but I wonder if there's a better way of doing it, I've looked at array_diff and such however if I have multiple stop words in the first array, array_diff only appears to remove the first occurrence. The focus is on speed and memory usage but speed more so. Thanks

  Read the article

< Previous Page | 272 273 274 275 276 277 278 279 280 281 282 283  | Next Page >