Search Results

Search found 35692 results on 1428 pages for 'build process'.

Page 515/1428 | < Previous Page | 511 512 513 514 515 516 517 518 519 520 521 522  | Next Page >

• looking for advise on importing excel into mysql with php

  - by Ole Media

  Alright, see if I can pick your brains from you all. I'm currently working on a project where all the information comes from different clients, the only thing in common is that the received data is done with excel. The excel spread sheet that they present is just a bunch of references and codes, and the problem than I'm facing is that I need the references and codes to be entered in certain format in order for the website to work. The perfect situation will be to go to each client and teach how I would need the data, but I can't do that because of the large number of clients, and more importantly I will be interrupting their work flow. Each client has its own codes and reference model and they are not willing to change their process The good news is that there is a standard pattern for the codes, but I'm talking close to 200 thousand codes with a bunch of combination. They way that we are currently solving the problem is that we have a person who checks each excel sheet received, runs a few macros, and manually fixes those codes in which the macro was not able to fix. The person that is doing this, is already burn out and frustrated and I would like to automatize this process with php. Suggestions?

  Read the article

• Activator.CreateInstance uses a huge amount of memory

  - by Marco

  I have been playing a bit with Silverlight and try to port my Silverlight 3.0 application to Silverlight 4.0. My application loads different XAP files and upon a user request create an instance of a Xaml user control and adds it to the main container, in a sort of MEF approach in order I can have an extensible and pluggable application. The application is pretty huge and to keep acceptable the performances and the initial loading I have built up some helper classes to load in the background all pages and user controls that might be used later on. On Silverlight 3.0 everything was running smoothly without any problem so far. Switching to SL 4.0 I have noticed that when the process approaches to create the instances of the user controls using Activator.CreateInstance, the layout freezes unexpectedly for a minute and sometimes for more. Looking at the task manager the memory usage of IE jumps from 50MB to 400MB and sometimes to 1.5 GB. If the process won't take that much the layout is rendered properly and the memory falls back to 50 MB. Otherwise everything crashes due to out of memory exeption. Does anybody encountered the same problem? Or has anybody a solution about this tricky issue?

  Read the article

• Recommendations for IPC between parent and child processes in .NET?

  - by Jeremy

  My .NET program needs to run an algorithm that makes heavy use of 3rd party libraries (32-bit), most of which are unmanaged code. I want to drive the CPU as hard as I can, so the code runs several threads in parallel to divide up the work. I find that running all these threads simultaneously results in temporary memory spikes, causing the process' virtual memory size to approach the 2 GB limit. This memory is released back pretty quickly, but occasionally if enough threads enter the wrong sections of code at once, the process crosses the "red line" and either the unmanaged code or the .NET code encounters an out of memory error. I can throttle back the number of threads but then my CPU usage is not as high as I would like. I am thinking of creating worker processes rather than worker threads to help avoid the out of memory errors, since doing so would give each thread of execution its own 2 GB of virtual address space (my box has lots of RAM). I am wondering what are the best/easiest methods to communicate the input and output between the processes in .NET? The file system is an obvious choice. I am used to shared memory, named pipes, and such from my UNIX background. Is there a Windows or .NET specific mechanism I should use?

  Read the article

• Information about PTE's (Page Table Entries) in Windows

  - by Patrick

  In order to find more easily buffer overflows I am changing our custom memory allocator so that it allocates a full 4KB page instead of only the wanted number of bytes. Then I change the page protection and size so that if the caller writes before or after its allocated piece of memory, the application immediately crashes. Problem is that although I have enough memory, the application never starts up completely because it runs out of memory. This has two causes: since every allocation needs 4 KB, we probably reach the 2 GB limit very soon. This problem could be solved if I would make a 64-bit executable (didn't try it yet). even when I only need a few hundreds of megabytes, the allocations fail at a certain moment. The second problem is the biggest one, and I think it's related to the maximum number of PTE's (page table entries, which store information on how Virtual Memory is mapped to physical memory, and whether pages should be read-only or not) you can have in a process. My questions (or a cry-for-tips): Where can I find information about the maximum number of PTE's in a process? Is this different (higher) for 64-bit systems/applications or not? Can the number of PTE's be configured in the application or in Windows? Thanks, Patrick PS. note for those who will try to argument that you shouldn't write your own memory manager: My application is rather specific so I really want full control over memory management (can't give any more details) Last week we had a memory overwrite which we couldn't find using the standard C++ allocator and the debugging functionality of the C/C++ run time (it only said "block corrupt" minutes after the actual corruption") We also tried standard Windows utilities (like GFLAGS, ...) but they slowed down the application by a factor of 100, and couldn't find the exact position of the overwrite either We also tried the "Full Page Heap" functionality of Application Verifier, but then the application doesn't start up either (probably also running out of PTE's)

  Read the article

• Where can I find a powerful, standards compliant, web-based interactive org chart API?

  - by Adam Soltys

  Hi, I'm looking to build an interactive web-based org chart for a large organization. I somewhat like the interface at ancestry.com where you can hover over people and pan/zoom around and click on different nodes to make them the root. Ideally, I'd like it if people could belong to multiple organizational entities like committees, working groups, etc. In other words the API should support graphs in general, not just trees. I'd like to be able to visually explode each organizational substructure into substituents by clicking on it, with a nice animation of the employees ballooning or spilling out so you can really interactively drill down through the organization. I found http://code.google.com/apis/visualization/documentation/gallery/orgchart.html but it looks a bit rudimentary. I know there are desktop tools like OrgPlus and Visio that can build static charts but I'm really looking for a free, web-based API with open standards-based output like SVG or HTML5 Canvas elements rather than Flash or some proprietary output. Something I can embed into a custom web application and style myself. Something interactive.

  Read the article

• Problem deleting .svn directories on Windows XP

  - by John L

  I don't seem to have this problem on my home laptop with Windows XP, but then I don't do much work there. On my work laptop, with Windows XP, I have a problem deleting directories when it has directories that contain .svn directories. When it does eventually work, I have the same issue emptying the Recycle bin. The pop-up window says "Cannot remove folder text-base: The directory is not empty" or prop-base or other folder under .svn This continued to happen after I changed config of TortoiseSVN to stop the TSVN cache process from running and after a reboot of the system. Multiple tries will eventually get it done. But it is a huge annoyance because there are other issues I'm trying to fix, so I'm hoping it is related. 'Connected Backup PC' also runs on the laptop and the real problem is that cygwin commands don't always work. So I keep thinking the dot files and dot directories have something to do with both problems and/or the backup or other process scanning the directories is doing it. But I've run out of ideas of what to try or how to identify the problem further.

  Read the article

• What happens when modifying Gemfile.lock directly?

  - by Mik378

  Since the second time of bundle install execution, dependencies are loaded from Gemfile.lock when Gemfile isn't changed. But I wonder how detection of changes is made between those two files. For instance, if I'm adding a new dependency directly into Gemfile.lock without adding it into Gemfile (as opposed to the best practice since Gemfile.lock is auto-generated from Gemfile), would a bundle install consider Gemfile as changed ? Indeed, does bundle install process compares the whole Gemfile and Gemfile.lock trees in order to detect changes? If it is, even if I'm adding a dependency directly to Gemfile.lock, Gemfile would be detected as changed (since different) and would re-erase Gemfile.lock (so losing the added dependency...) What is the process of bundle install since the launch for the second time ? To be more clear, my question is: Are changes based only from Gemfile ? That means bundler would keep a Gemfile snapshot of every bundle install execution number N and merely compares it to the bundle install execution N+1 ? Or none snapshot are created in bundler memory and bundler makes a comparison with Gemfile.lock each time to detect if Gemfile must be considered as changed.

  Read the article

• Is there a better way to keep track of session variable creation/access throughout different pages?

  - by Brandon

  Here's what I am working on. At my website I have multiple processes with each one containing multiple steps. Now in one of the processes, there is an error checking routine executed before proceeding to the next step of that process. A session var is set indicating the error status and it will either redirect back to the referrer or display the next page's contents. Now this kind of functionality, I believe, is common throughout web development. The issue that is occurring is that session vars are left around and are not being cleaned up properly. At times this introduces undesired behavior. My website is growing and I find that I am requiring more and more session vars to keep track of different system and error states. So I was thinking about creating a kind of "session variable keeper" to keep track of session var usage. The idea is fairly simple. It will have the notion of a context (e.g. registration process) and allow access to a predefined set of session vars within that context. In addition, the var and context will be paired with an action to proceed to some form of event handling. So if you haven't noticed I'm new to web development. Any thoughts or comments on the idea that I am proposing would be greatly appreciated. The back-end is written in PHP/MySQL.

  Read the article

• What is the proper syntax for getting a Makefile to print the output directory of one of its output zip files?

  - by 9exceptionThrower9

  I'm trying to edit an Android Makefile in the hopes of getting it to print out the directory (path) location of one the ZIP files it creates. Ideally, since the build process is long and does many things, I would like for it print out the pathway to the ZIP file to a text file in a different directory I can access later: Pseudo-code idea: # print the desired pathway to output file print(getDirectoryOf(variable-name.zip)) > ~/Desktop/location_of_file.txt The Makefile snippet where I would like to insert this new bit of code is shown below. I am interested in finding the directory of $(name).zip (that is specific file I want to locate): # ----------------------------------------------------------------- # A zip of the directories that map to the target filesystem. # This zip can be used to create an OTA package or filesystem image # as a post-build step. # name := $(TARGET_PRODUCT) ifeq ($(TARGET_BUILD_TYPE),debug) name := $(name)_debug endif name := $(name)-target_files-$(FILE_NAME_TAG) intermediates := $(call intermediates-dir-for,PACKAGING,target_files) BUILT_TARGET_FILES_PACKAGE := $(intermediates)/$(name).zip $(BUILT_TARGET_FILES_PACKAGE): intermediates := $(intermediates) $(BUILT_TARGET_FILES_PACKAGE): \ zip_root := $(intermediates)/$(name) # $(1): Directory to copy # $(2): Location to copy it to # The "ls -A" is to prevent "acp s/* d" from failing if s is empty. define package_files-copy-root if [ -d "$(strip $(1))" -a "$$(ls -A $(1))" ]; then \ mkdir -p $(2) && \ $(ACP) -rd $(strip $(1))/* $(2); \ fi endef

  Read the article

• Why is Log4Net not creating log file in production?

  - by uriDium

  I am using VS2005, a website project, a web deployment project and Log4Net. I can use logging when I am developing locally. I can see the log files and everything is fine. When I build my website, (using the web deployment project), I use the deploy as a single DLL option. When I then check the locations of where my log files should be I cannot see any files. Is there a way to troubleshoot this. I don't think adding the debug value to the App Settings will help because I don't have a console because it is a website. EDIT I don't want the 150 rep to go to waste so one last time. I compared the internal trace from my dev environment to the trace from the production. My dev environment trace shows the call the Xml Configurator where the production one does not. I have code in the global.asax on application_start() method. I put debug code in there and it is getting called in dev but not in production. I think this is where the web deployment project is causing some issues. Does the global.asax get compiled into the single DLL? When I do a build in the deployment directory I see a global.compiled file. Must this go into the bin folder in production? Or is the global.asax code in the single DLL? Having both in the bin folder or the just the DLL didn't change anything.

  Read the article

• Think this is a naming problem

  - by RussP

  Must be really dumb today - sorry in advance; anyhow have this unordered list <ul> <li><div class="openuserform" >Info</div> <div class="userform"></div></li> <li><div class="openuserform" >Appearence</div> <div class="userform"></div></li> <li><div class="openuserform" >Pages</div> <div class="userform"></div></li> <li><div class="openuserform" >Services</div> <div class="userform"></div></li> <li><div class="openuserform" >Community</div> <div class="userform"></div></li> </ul> which on click <div class="openuserform" > I want to load a separate form e.g. $('.openusersform').live('click',(function(){ $('.userform').load('form page.php'); }); OK I can get the forms to load in the right div using $(this).next('.userform').show(); $('.userform').load('form page.php'); but it's very ugly (I think) and I can only every get the first form to process propery. It is laid out like (ul,li etc.) this so I can have each loaded form aligned under the relevant li. But I think there has to be a better way as I do not seem to get any if(){} stemnents working to process the forms i.e. if(form1){ $.ajax etc } if (form2) {more ajax} Suggestions please - thanks

  Read the article

• problem in case of window service

  - by prateeksaluja20

  Hello friends, i made a windows service & add project installer.in which only contain this code. System.Diagnostics.Process.Start(@"C:\Windows\system32\notepad.exe"); inside the timer tick event & interval is 60 sec.i just wanted to try to run Windows service. 1st-serviceProcessInstaller1 i have been changed its account setting as local system. 2nd-serviceInstaller1 in this case i have been changed its start up type as Automatic. then i create a setup add another project then right click add project output then add primary output then press ok. then go to Right click on project-view-custom Action-right click on Install-Add custom Action-select Application folder & add primary output.the same thing done for all the remaining options like commit,rollback,uninstall. after that i build the setup it build succesfully then i install the setup it installed properly into program file n create one .exe file n one Instalfile. but problem is that when i search the service into "services.msc" the service is not there. means service is not showing there.i tried but not getting the ans.plz help me to solve this problem.

  Read the article

• Using AJAX in Rails: How do I change a button as soon as it's clicked?

  - by sdc

  Hey! I'm teaching myself Ruby, and have been stuck on this for a couple days: I'm currently using MooTools-1.3-compat and Rails 3. I'd like to replace one button (called "Follow") with another (called "Unfollow") as soon as someone clicks on it. I'm using :remote = true and have a file ending in .js.erb that's being called...I just need help figuring out what goes in this .js file The "Follow" button is in a div with id="follow_form", but there are many buttons on the page, and they all have an id = "follow_form"...i.e. $("follow_form").set(...) replaces the first element and that's not correct. I need help replacing the button that made the call. I looked at this tutorial, but the line below doesn't work for me. Could it be because I'm using MooTools instead of Prototype? $("follow_form").update("<%= escape_javascript(render('users/unfollow')) %") ps. This is what I have so far, and this works: in app/views/shared: <%= form_for current_user.subscriptions.build(:event => @event), :remote => true do |f| %> <div><%= f.hidden_field :event %></div> <div class="actions"><%= f.submit "Follow" %></div> <% end %> in app/views/events/create.js.erb alert("follow!"); //Temporary...this is what I'm trying to replace *in app/controllers/subscriptions_controller.rb* def create @subscription = current_user.subscriptions.build(params[:subscription]) @subscription.save respond_to do |format| format.html { redirect_to(..) } format.js {render :layout} end Any help would be greatly, greatly appreciated!

  Read the article

• How can I write to a single xml file from two programs at the same time?

  - by Tom Bushell

  We recently started working with XML files, after many years of experience with the old INI files. My coworker found some CodeProject sample code that uses System.Xml.XmlDocument.Save. We are getting exceptions when two programs try to write to the same file at the same time. System.IO.IOException: The process cannot access the file 'C:\Test.xml' because it is being used by another process. This seems obvious in hindsight, but we had not anticipated it because accessing INI files via the Win32 API does not have this limitation. I assume there's some arbitration done by the Win32 calls that work at a higher level than the the XmlDocument.Save method. I'm hoping there are higher level XML routines somewhere in the .Net library that work similarily to the Win32 functions, but don't know where to start looking. Or maybe we can set up our file access permissions to allow multiple programs to write to the same file? Time is short (like almost all SW projects), and if we can't find a solution quickly, we'll have to hold our noses and go back to INI files.

  Read the article

• How do I redirect standard output to a file in Perl? [closed]

  - by rockyurock

  I want to send standard output to the file "my_output.txt" but failed. Here's the output: inside value loop ------------------------------------------------------------ Server listening on UDP port 5001 Receiving 1470 byte datagrams UDP buffer size: 108 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.16.2 port 5001 connected with 192.168.16.1 port 3189 [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 3] 0.0- 5.0 sec 2.14 MBytes 3.61 Mbits/sec 0.369 ms 0/ 1528 (0%) inside value loop3 clue1 clue2 inside value loop4 one iperf completed *************************************** When I enable the local *STDOUT; in below code then I could see the above output on command prompt display (ofcourse server is sending some data): my $file = 'my_output.txt'; use Win32::Process; print"inside value loop\n"; # redirect stdout to a file #local *STDOUT; open STDOUT, '>', $file or die "can't redirect STDOUT to <$file> $!"; Win32::Process::Create(my $ProcessObj, "D:\\IOT_AUTOMATION_UTILITY\\_SATURDAY_09-04-10\\adb_cmd.bat", "adb shell /data/app/iperf -u -s -p 5001", 0, NORMAL_PRIORITY_CLASS, ".") || die ErrorReport(); #$alarm_time = $IPERF_RUN_TIME+10; #20sec #$ProcessObj->Wait(40); #print"inside value loop2\n"; #sleep $alarm_time; sleep 40; $ProcessObj->Kill(0); sub ErrorReport{ print Win32::FormatMessage( Win32::GetLastError() ); }

  Read the article

• How do I stop Ant from hanging after executing a java program that attempted to interrupt a thread (and failed) and continued?

  - by Zugwalt

  I have Ant build and execute a java program. This program tries to do something that sometimes hangs, so we execute it in a thread. actionThread.start(); try { actionThread.join(10000); } catch (InterruptedException e) { System.out.println("InterruptedException: "+e.getMessage()); } if (actionThread.isAlive()) { actionThread.interrupt(); System.out.println("Thread timed out and never died"); } The ant call looks like this: <java fork="true" failonerror="yes" classname="myPackage.myPathName" classpath="build"> <arg line=""/> <classpath> <pathelement location="bin" /> <fileset dir="lib"> <include name="**/*.jar"/> </fileset> </classpath> </java> And when this runs I see the "Thread timed out and never died" statement, and I also see the main program finish execution, but then Ant just hangs. Presumably it is waiting for the child threads to finish, but they never will. How can I have Ant be done once it is done executing main() and just kill or ignore dead threads?

  Read the article

• How change encoding ?

  - by simply denis

  I need convert or set encoding windows-1251 Process p = new Process(); StreamWriter sw; StreamReader sr; StreamReader err; ProcessStartInfo psI = new ProcessStartInfo("cmd"); psI.UseShellExecute = false; psI.RedirectStandardInput = true; psI.RedirectStandardOutput = true; psI.RedirectStandardError = true; psI.CreateNoWindow = true; p.StartInfo = psI; p.Start(); sw = p.StandardInput; sr = p.StandardOutput; err = p.StandardError; sw.AutoFlush = true; if (tbComm.Text != "") sw.WriteLine(tbComm.Text); else //execute default command sw.WriteLine("dir \\"); sw.Close(); textBox1.Text = sr.ReadToEnd();// this not support russian word. I need convert or set encoding windows-1251 textBox1.Text += err.ReadToEnd();

  Read the article

• Do I lose the benefits of macro recording if I develop Excel apps in Visual Studio?

  - by DanM

  I've written lots of Excel macros in the past using the following development process: Record a macro. Open the VBA editor. Edit the macro. I'm now experimenting with a Visual Studio 2008 "Excel 2007 Add-In" project (C#), and I'm wondering if I will have to give up this development process. Questions: I know I can still record macros using Excel, but is there any way to access the resulting code in Visual Studio? Or do I just have to copy and paste then C#-ize it? What happens with my "Personal Macro Workbook"? Can I use the macros I have stored in there within C#? Or is there some way to convert them to C#? If there is some support for opening and editing VBA macros in Visual Studio, can you provide a very brief summary of how it works or point me to a good reference? Do you have any other tips for transitioning from writing macros in VBA using Excel's built-in editor to writing them in C# with Visual Studio?

  Read the article

• "Ghost" values in PHP/Smarty.

  - by Kyle Sevenoaks

  I've been working on a site for a while changing the layout and skin of a webshop checkout process. I've noticed that if you go all the way through the process until the last page, then click the link to go back to the view products page, the delivery method price displays underneath the navigation buttons, until you refresh and it goes away again. I've downloaded both sourced from the browser (Chrome, but this bug applies to all browsers) and used a file difference tool to display the differences, the result being only: < error.html vs > normal.html 34c34 < <link href="gzip.php?file=167842c1496093fbcd391b41cf7b03da.css&time=1272272181" rel="Stylesheet" type="text/css"/> --- > <link href="gzip.php?file=167842c1496093fbcd391b41cf7b03da.css&time=1272272348" rel="Stylesheet" type="text/css"/> Which is just the way it zips up the CSS stylesheets. (afaik) Has anyone ever encountered such a problem, or anything similar? Normal: Error: I can't even hazard a guess as to what is causing this, at all. I've searched over Google for anything and come up with nothing. What could be causing this? The site in question is Euroworker.no. HTML @ Pastebin. Smarty snippet: {if !$CANONICAL} {canonical}{self}{/canonical} {/if} <link rel="canonical" href="{$CANONICAL}" /> <!-- Css includes --> {includeCss file="frontend/Frontend.css"} {includeCss file="backend/stat.css"} {if {isRTL}} {includeCss file="frontend/FrontendRTL.css"} {/if} {compiledCss glue=true nameMethod=hash} <!--[if lt IE 8]> <link href="stylesheet/frontend/FrontendIE.css" rel="Stylesheet" type="text/css"/> {if $ieCss} <link href="{$ieCss}" rel="Stylesheet" type="text/css"/> {/if} <![endif]--> Thanks.

  Read the article

• Implimenting Zend MVC for my existing site-first step?

  - by Joel

  Hi guys, OK-newbie question here. I'll try not to bombard SO with lots of questions-and hopefully this first one will show me the method I'll need to follow for subsequent conversions. I have a web-based calendar system that I developed, but it was coded for me procedurally (using PHP). I'm now working on learning OO and wanting to integrate this site into my localhost Zend Framework and slowly start converting parts to OO and the Zend Framework MVC process in particular. As I've said before, I understand that this will be a slow process, and when I'm done, I still probably won't have anything as OO friendly as if I had rewritten it from scratch, but I'd like to use this as a learning experience. So, I have dropped the whole site into my localhose/zend/Public folder, and everything is showing up great and linking to the database, etc. My question is-what would be the easiest first component to switch over to the MVC model? This site has a bit of everything-forms, login, authentication, some jQuery, etc. Can anyone point to a tutorial that would address what I'm trying to do? If indeed, a form would be one of the simpler things to switch, can someone walk me through those changes? Another idea is changing over all the header info, etc? Thanks for any pointers on where to start! EDIT: Also, I understand that SO is mainly for specific coding questions-I'm happy to share specific code, once I have an idea about which section to tackle first...

  Read the article

• Does IE completely ignore cache control headers for AJAX requests?

  - by Joshua Hayworth

  Hello there, I've got, what I would consider, a simple test web site. A single page with a single button. Here is a copy of the source I'm working with if you would like to download it and play with it. When that button is clicked, it creates a JavaScript timer that executes once a second. When the timer function is executed, An AJAX call is made to retrieve a text value. That text value is then placed into the DOM. What's my problem? IE Caching. Crack open Task Manager and watch what happens to the iexplorer.exe process (IE 8.0.7600.16385 for me) while the timer in that page is executing. See the memory and handle count getting larger? Why is that happening when, by all accounts, I have caching turned off. I've got the jQuery cache option set to false in $.ajaxSetup. I've got the CacheControl header set to no-cache and no-store. The Expires header is set to DateTime.Now.AddDays(-1). The headers are set in both the page code-behind as well as the HTTP Handler's response. Anybody got any ideas as to how I could prevent IE from caching the results of the AJAX call? Here is what the iexplorer.exe process looks like in ProcessMonitor. I believe that the activity shown in this picture is exactly what I'm attempting to prevent.

  Read the article

• How to load/save C++ class instance (using STL containers) to disk

  - by supert

  I have a C++ class representing a hierarchically organised data tree which is very large (~Gb, basically as large as I can get away with in memory). It uses an STL list to store information at each node plus iterators to other nodes. Each node has only one parent, but 0-10 children. Abstracted, it looks something like: struct node { public: node_list_iterator parent; // iterator to a single parent node double node_data_array[X]; map<int,node_list_iterator> children; // iterators to child nodes }; class strategy { private: list<node> tree; // hierarchically linked list of nodes struct some_other_data; public: void build(); // build the tree void save(); // save the tree from disk void load(); // load the tree from disk void use(); // use the tree }; I would like to implement the load() and save() to disk, and it should be fairly fast, however the obvious problems are: I don't know the size in advance; The data contains iterators, which are volatile; My ignorance of C++ is prodigious. Could anyone suggest a pure C++ solution please?

  Read the article

• Scalably processing large amount of comlpicated database data in PHP, many times a day.

  - by Eph

  I'm soon to be working on a project that poses a problem for me. It's going to require, at regular intervals throughout the day, processing tens of thousands of records, potentially over a million. Processing is going to involve several (potentially complicated) formulas and the generation of several random factors, writing some new data to a separate table, and updating the original records with some results. This needs to occur for all records, ideally, every three hours. Each new user to the site will be adding between 50 and 500 records that need to be processed in such a fashion, so the number will not be steady. The code hasn't been written, yet, as I'm still in the design process, mostly because of this issue. I know I'm going to need to use cron jobs, but I'm concerned that processing records of this size may cause the site to freeze up, perform slowly, or just piss off my hosting company every three hours. I'd like to know if anyone has any experience or tips on similar subjects? I've never worked at this magnitude before, and for all I know, this will be trivial to the server and not pose much of an issue. As long as ALL records are processed before the next three hour period occurs, I don't care if they aren't processed simultaneously (though, ideally, all records belonging to a specific user should be processed in the same batch), so I've been wondering if I should process in batches every 5 minutes, 15 minutes, hour, whatever works, and how best to approach this (and make it scalable in a way that is fair to all users)?

  Read the article

• Communicating with a running python daemon

  - by hanksims

  I wrote a small Python application that runs as a daemon. It utilizes threading and queues. I'm looking for general approaches to altering this application so that I can communicate with it while it's running. Mostly I'd like to be able to monitor its health. In a nutshell, I'd like to be able to do something like this: python application.py start # launches the daemon Later, I'd like to be able to come along and do something like: python application.py check_queue_size # return info from the daemonized process To be clear, I don't have any problem implementing the Django-inspired syntax. What I don't have any idea how to do is to send signals to the daemonized process (start), or how to write the daemon to handle and respond to such signals. Like I said above, I'm looking for general approaches. The only one I can see right now is telling the daemon constantly log everything that might be needed to a file, but I hope there's a less messy way to go about it. UPDATE: Wow, a lot of great answers. Thanks so much. I think I'll look at both Pyro and the web.py/Werkzeug approaches, since Twisted is a little more than I want to bite off at this point. The next conceptual challenge, I suppose, is how to go about talking to my worker threads without hanging them up. Thanks again.

  Read the article

• Many users, many cpus, no delays. Good for cloud?

  - by Eric

  I wish to set up a CPU-intensive time-important query service for users on the internet. A usage scenario is described below. Is cloud computing the right way to go for such an implementation? If so, what cloud vendor(s) cater to this type of application? I ask specifically, in terms of: 1) pricing 2) latency resulting from: - slow CPUs, instance creations, JIT compiles, etc.. - internal management and communication of processes inside the cloud (e.g. a queuing process and a calculation process) - communication between cloud and end user 3) ease of deployment A usage scenario I am expecting is: - A typical user sends a query (XML of size around 1K) once every 30 seconds on average. - Each query requires a numerical computation of average time 0.2 sec and max time 1 sec on a 1 GHz Pentium. The computation requires no data other than the query itself and is performed by the same piece of code each time. - The delay a user experiences between sending a query and receiving a response should be on average no more than 2 seconds and in general no more than 5 seconds. - A background save to a DB of the response should occur (not time critical) - There can be up to 30000 simultaneous users - i.e., on average 1000 queries a second, each requiring an average 0.2 sec calculation, so that would necessitate around 200 CPUs. Currently I'm look at GAE Java (for quicker deployment and less IT hassle) and EC2 (Speed and price optimization) as options. Where can I learn more about the right way to set ups such a system? past threads, different blogs, books, etc.. BTW, if my terminology is wrong or confusing, please let me know. I'd greatly appreciate any help.

  Read the article

< Previous Page | 511 512 513 514 515 516 517 518 519 520 521 522  | Next Page >