Search Results

Search found 23098 results on 924 pages for 'multiple processes'.

Page 654/924 | < Previous Page | 650 651 652 653 654 655 656 657 658 659 660 661  | Next Page >

  • Java: design for using many executors services and only few threads

    - by Guillaume
    I need to run in parallel multiple threads to perform some tests. My 'test engine' will have n tests to perform, each one doing k sub-tests. Each test result is stored for a later usage. So I have n*k processes that can be ran concurrently. I'm trying to figure how to use the java concurrent tools efficiently. Right now I have an executor service at test level and n executor service at sub test level. I create my list of Callables for the test level. Each test callable will then create another list of callables for the subtest level. When invoked a test callable will subsequently invoke all subtest callables test 1 subtest a1 subtest ...1 subtest k1 test n subtest a2 subtest ...2 subtest k2 call sequence: test manager create test 1 callable test1 callable create subtest a1 to k1 testn callable create subtest an to kn test manager invoke all test callables test1 callable invoke all subtest a1 to k1 testn callable invoke all subtest an to kn This is working fine, but I have a lot of new treads that are created. I can not share executor service since I need to call 'shutdown' on the executors. My idea to fix this problem is to provide the same fixed size thread pool to each executor service. Do you think it is a good design ? Do I miss something more appropriate/simple for doing this ?

    Read the article

  • Phonegap Android application exiting - but not really - when home button is pressed

    - by Lucas T
    I have created an Phonegap 1.5/Android application. My client reports that, when he leaves the app using the Home button, and then relaunches it using the app icon, the app relaunches from the start instead of resuming. However, when he holds the home button, the app appears in the running apps, and when he accesses the app through this menu, the app resumes in the expected way. I thought this could be linked to the app being automatically closed by the OS due to a lack of memory, but if that was the case the app shouldn't appear in the running apps. I could not reproduce the bug on my Sony Ericsson XPERIA with Android 2.3.4, the client has experienced this behaviour on a Motorola Defy and on another phone (i'll add the reference of the other phone and the OS versions as soon as I get them). The initialization process of the app is declared this way : window.addEventListener('load', function(){ document.addEventListener('deviceready', _onDeviceReady, false); }, false); Could this be fixed by attaching the processes to other events (although I doubt it, the app really seems to be relaunched from the start) ? Is there a declaration to make in the Android Manifest to prevent this behavior ? Is that a known bug in some Android phones/versions ?

    Read the article

  • Protecting critical sections based on a condition in C#

    - by NAADEV
    Hello, I'm dealing with a courious scenario. I'm using EntityFramework to save (insert/update) into a SQL database in a multithreaded environment. The problem is i need to access database to see whether a register with a particular key has been already created in order to set a field value (executing) or it's new to set a different value (pending). Those registers are identified by a unique guid. I've solved this problem by setting a lock since i do know entity will not be present in any other process, in other words, i will not have same guid in different processes and it seems to be working fine. It looks something like that: static readonly object LockableObject = new object(); static void SaveElement(Entity e) { lock(LockableObject) { Entity e2 = Repository.FindByKey(e); if (e2 != null) { Repository.Insert(e2); } else { Repository.Update(e2); } } } But this implies when i have a huge ammount of requests to be saved, they will be queued. I wonder if there is something like that (please, take it just as an idea): static void SaveElement(Entity e) { (using ThisWouldBeAClassToProtectBasedOnACondition protector = new ThisWouldBeAClassToProtectBasedOnACondition(e => e.UniqueId) { Entity e2 = Repository.FindByKey(e); if (e2 != null) { Repository.Insert(e2); } else { Repository.Update(e2); } } } The idea would be having a kind of protection that protected based on a condition so each entity e would have its own lock based on e.UniqueId property. Any idea?

    Read the article

  • What's the best way of accessing a DRb object (e.g. Ruby Queue) from Scala (and Java)?

    - by Tom Morris
    I have built a variety of little scripts using Ruby's very simple Queue class, and share the Queue between Ruby and JRuby processes using DRb. It would be nice to be able to access these from Scala (and maybe Java) using JRuby. I've put together something Scala and the JSR-223 interface to access jruby-complete.jar. import javax.script._ class DRbQueue(host: String, port: Int) { private var engine = DRbQueue.factory.getEngineByName("jruby") private var invoker = engine.asInstanceOf[Invocable] engine.eval("require \"drb\" ") private var queue = engine.eval("DRbObject.new(nil, \"druby://" + host + ":" + port.toString + "\")") def isEmpty(): Boolean = invoker.invokeMethod(this.queue, "empty?").asInstanceOf[Boolean] def size(): Long = invoker.invokeMethod(this.queue, "length").asInstanceOf[Long] def threadsWaiting: Long = invoker.invokeMethod(this.queue, "num_waiting").asInstanceOf[Long] def offer(obj: Any) = invoker.invokeMethod(this.queue, "push", obj.asInstanceOf[java.lang.Object]) def poll(): Any = invoker.invokeMethod(this.queue, "pop") def clear(): Unit = { invoker.invokeMethod(this.queue, "clear") } } object DRbQueue { var factory = new ScriptEngineManager() } (It conforms roughly to java.util.Queue interface, but I haven't declared the interface because it doesn't implement the element and peek methods because the Ruby class doesn't offer them.) The problem with this is the type conversion. JRuby is fine with Scala's Strings - because they are Java strings. But if I give it a Scala Int or Long, or one of the other Scala types (List, Set, RichString, Array, Symbol) or some other custom type. This seems unnecessarily hacky: surely there has got to be a better way of doing RMI/DRb interop without having to use JSR-223 API. I could either make it so that the offer method serializes the object to, say, a JSON string and takes a structural type of only objects that have a toJson method. I could then write a Ruby wrapper class (or just monkeypatch Queue) to would parse the JSON. Is there any point in carrying on with trying to access DRb from Java/Scala? Might it just be easier to install a real message queue? (If so, any suggestions for a lightweight JVM-based MQ?)

    Read the article

  • Is there a way to validate the presence of Javadoc and/or inline code comments?

    - by Chris Aldrich
    We are trying to put quality code processes in place for a large project I am working on. Right now a lot of developers are not putting in Javadoc or in-line code comments into their code. Ok right now. But it will severely hurt us in the very near future. We are using Maven 2.0.9 as our build tool, as well as Hudson for Continuous Integration. We are using Subversion as our source versioning tool/code repository, Rational Application Developer and Rational Softare Architect (essentially Eclipse) 7.5.1 as our IDE's, and then Subclipse as our Eclipse plug-in to connect to SVN. Is there a plug-in or a way to validate that a developer put in Javadoc and/or in-line code comments in order to allow a commit to SVN? This isn't intended to be a substitute for good code reviews, but merely a help to make sure that developers are reminded to add this documentation before committing. We are still intending on conducting code reviews that would also review documentation. Has anyone found any plug-ins for something like this? Any links? Any ideas?

    Read the article

  • Yeoman 'grunt test' fails on clean project with 'port already in use'

    - by XMLilley
    With: Mac OS 10.8.4 Node 0.10.12 npm 1.3.1 grunt-cli 0.1.9 yo 1.0.0-rc.1 bower 0.9.2 [email protected] I encounter the following error with a clean yo angular project, followed by grunt server then grunt test: Running "connect:test" (connect) task Fatal error: Port 9000 is already in use by another process. I'm new to Yeoman and am stumped. I've deleted my original project and created a new one in a fresh folder just to make sure I wasn't overlooking any invisible configs. I restarted the machine to make sure I wasn't running any temporary server processes I had forgotten about. After all attempts, the basic server starts fine, attaches to Chrome, and the watcher updates the browser on any changes. (Notably, the server is running on 9000, which seems odd for the test-runner to also be trying to use 9000.) But I get that same error on attempting to start the test runner. Is this something I can fix, or an issue I should report to the Yeoman team? Thanks.

    Read the article

  • Limit CPU usage of a process

    - by jb
    I have a service running which periodically checks a folder for a file and then processes it. (Reads it, extracts the data, stores it in sql) So I ran it on a test box and it took a little longer thaan expected. The file had 1.6 million rows, and it was still running after 6 hours (then I went home). The problem is the box it is running on is now absolutely crippled - remote desktop was timing out so I cant even get on it to stop the process, or attach a debugger to see how far through etc. It's solidly using 90%+ CPU, and all other running services or apps are suffering. The code is (from memory, may not compile): List<ItemDTO> items = new List<ItemDTO>(); using (StreamReader sr = fileInfo.OpenText()) { while (!sr.EndOfFile) { string line = sr.ReadLine() try { string s = line.Substring(0,8); double y = Double.Parse(line.Substring(8,7)); //If the item isnt already in the collection, add it. if (items.Find(delegate(ItemDTO i) { return (i.Item == s); }) == null) items.Add(new ItemDTO(s,y)); } catch { /*Crash*/ } } return items; } - So I am working on improving the code (any tips appreciated). But it still could be a slow affair, which is fine, I've no problems with it taking a long time as long as its not killing my server. So what I want from you fine people is: 1) Is my code hideously un-optimized? 2) Can I limit the amount of CPU my code block may use? Cheers all

    Read the article

  • Custom UIButton Memory Management in dealloc

    - by ddawber
    I am hoping to clarify the processes going on here. I have created a subclass of UIButton whose init method looks like this: - (id)initWithTitle:(NSString *)title frame:(CGRect)btnFrame { self = [UIButton buttonWithType:UIButtonTypeCustom]; [self setTitle:title forState:UIControlStateNormal]; self.frame = btnFrame; return self; } In my view controller I am creating one of these buttons and adding it as a subview: myButton = [[CustomButton alloc] initWithTitle:@"Title" frame:someFrame]; [self.view addSubview:myButton]; In the view controller's dealloc method I log the retain count of my button: - (void)dealloc { NSLog(@"RC: %d", [myButton retainCount]); //RC = 2 [super dealloc]; NSLog(@"RC: %d", [myButton retainCount]); //RC = 1 } The way I understand it, myButton is not actually retained, even though I invoked it using alloc, because in my subclass I created an autorelease button (using buttonWithType:). In dealloc, does this mean that, when dealloc is called the superview releases the button and its retain count goes down to 1? The button has not yet been autoreleased? Or do I need to get that retain count down to zero after calling [super dealloc]? Cheers.

    Read the article

  • Performing measures within the execution of a c++ code every t milliseconds

    - by user506901
    Given a while loop and the function ordering as follows: int k=0; int total=100; while(k<total){ doSomething(); if(approx. t milliseconds elapsed) { measure(); } ++k; } I want to perform 'measure' every t-th milliseconds. However, since 'doSomething' can be close to the t-th millisecond from the last execution, it is acceptable to perform the measure after approximately t milliseconds elapsed from the last measure. My question is: how could this be achieved? One solution would be to set timer to zero, and measure it after every 'doSomething'. When it is withing the acceptable range, I perform measures, and reset. However, I'm not which c++ function I should use for such a task. As I can see, there are certain functions, but the debate on which one is the most appropriate is outside of my understanding. Note that some of the functions actually take into account the time taken by some other processes, but I want my timer to only measure the time of the execution of my c++ code (I hope that is clear). Another thing is the resolution of the measurements, as pointed out below. Suppose the medium option of those suggested.

    Read the article

  • Some images fails to load on Windows Server 2008

    - by Guffa
    I have an application running on a Windows Server 2008, that is processing uploaded images. Currently it successfully processes about 8000 images per day, creating 11 different sizes of each image. The problem that I have is that sometimes the application fails to load some images, I get the error "System.Runtime.InteropServices.ExternalException: A generic error occurred in GDI+.". The upload only accepts files with a JPEG extension (jpg/jpeg/jpe) or with a JPEG MIME type, and from what I can tell those images are really JPEG images. If I look at the image file in windows explorer on the server, it can successfully extract the thumbnail from the file, but if I try to open it, I get the error message "This is not a valid bitmap file, or it's format is not currently supported." from Paint. If I copy the image to my own computer, running Windows 7, there is no problem opening the image. It works in Paint, Windows Photo Viewer, Adobe Bridge, and Photoshop. If I try to load the image using Image.FromStream the same way as in the application running on the server, it loads just fine. (I have copied the file back to the server, and it still doesn't work, so there is nothing in the copying process that changes it.) When I look at the image information in Bridge, I see that the images are created using Picasa 3.0, but other than that I can't see anything special about them. I haven't yet found anyone having the same problem, or any known problems like this with the Picasa application. Has anyone had any similar problem, or know if there is something special about images created using Picasa? Is there any image codec that needs installing on the server to handle all kinds of JPEG images?

    Read the article

  • Why does the JSF action tag handler in JSP invoke rendering immediately after creation?

    - by Pentius
    Dear fellows, I read the article "Improving JSF by Dumping JSP" from Hans Bergsten. There I read the following: The JSP container processes the page and invokes the JSF action tag handlers as they are encountered. A JSF tag handler looks for the JSF component it represents in the component tree. If it can't find the component, it creates it and adds it to the component tree. It then asks the component to render itself. and furthermore On the first request, the action creates its component and asks it to render itself. I understand that the immediate rendering after the creation of the component is the problem here (The reference to the input component can't be resolved in the example). That's one point, why JSF doesn't fit with JSP. But it reads as if the action tag handler itself would ask the component to render. Or is it JSP that triggers the rendering directly after the action tag handler created the component. If it is the action tag handler, I don't understand, why this is the fault of JSP. What is different here than from JSF intended? Thanks for your help, I need this for my thesis.

    Read the article

  • referencing part of the composite primary key

    - by Zavael
    I have problems with setting the reference on database table. I have following structure: CREATE TABLE club( id INTEGER NOT NULL, name_short VARCHAR(30), name_full VARCHAR(70) NOT NULL ); CREATE UNIQUE INDEX club_uix ON club(id); ALTER TABLE club ADD CONSTRAINT club_pk PRIMARY KEY (id); CREATE TABLE team( id INTEGER NOT NULL, club_id INTEGER NOT NULL, team_name VARCHAR(30) ); CREATE UNIQUE INDEX team_uix ON team(id, club_id); ALTER TABLE team ADD CONSTRAINT team_pk PRIMARY KEY (id, club_id); ALTER TABLE team ADD FOREIGN KEY (club_id) REFERENCES club(id); CREATE TABLE person( id INTEGER NOT NULL, first_name VARCHAR(20), last_name VARCHAR(20) NOT NULL ); CREATE UNIQUE INDEX person_uix ON person(id); ALTER TABLE person ADD PRIMARY KEY (id); CREATE TABLE contract( person_id INTEGER NOT NULL, club_id INTEGER NOT NULL, wage INTEGER ); CREATE UNIQUE INDEX contract_uix on contract(person_id); ALTER TABLE contract ADD CONSTRAINT contract_pk PRIMARY KEY (person_id); ALTER TABLE contract ADD FOREIGN KEY (club_id) REFERENCES club(id); ALTER TABLE contract ADD FOREIGN KEY (person_id) REFERENCES person(id); CREATE TABLE player( person_id INTEGER NOT NULL, team_id INTEGER, height SMALLINT, weight SMALLINT ); CREATE UNIQUE INDEX player_uix on player(person_id); ALTER TABLE player ADD CONSTRAINT player_pk PRIMARY KEY (person_id); ALTER TABLE player ADD FOREIGN KEY (person_id) REFERENCES person(id); -- ALTER TABLE player ADD FOREIGN KEY (team_id) REFERENCES team(id); --this is not working It gives me this error: Error code -5529, SQL state 42529: a UNIQUE constraint does not exist on referenced columns: TEAM in statement [ALTER TABLE player ADD FOREIGN KEY (team_id) REFERENCES team(id)] As you can see, team table has composite primary key (club_id + id), the person references club through contract. Person has some common attributes for player and other staff types. One club can have multiple teams. Employed person has to have a contract with a club. Player (is the specification of person) - if emplyed - can be assigned to one of the club's teams. Is there better way to design my structure? I thought about excluding the club_id from team's primary key, but I would like to know if this is the only way. Thanks. UPDATE 1 I would like to have the id as team identification only within the club, so multiple teams can have equal id as long as they belong to different clubs. Is it possible? UPDATE 2 updated the naming convention as adviced by philip Some business rules to better understand the structure: One club can have 1..n teams (Main squad, Reserve squad, Youth squad or Team A, Team B... only team can play match, not club) One team belongs to one club only A player is type of person (other types (staff) are scouts, coaches etc so they do not need to belong to specific team, just to the club, if employed) Person can have 0..1 contract with 1 club (that means he is employed or unemployed) Player (if employed) belongs to one team of the club Now thinking about it - moving team_id from player to contract would solve my problem, and it could hold the condition "Player (if employed) belongs to one team of the club", but it would be redundant for other staff types. What do you think?

    Read the article

  • Passing an address inside a WordPress post to a Google Map elsewhere on the Page

    - by ael_ecurai
    Background: My client is building their own WordPress site (using a purchased feature-rich theme), and I'm modifying a child theme as necessary to achieve customizations she wants. The theme comes with a Page template that includes a full-width Google Map across the top, which pulls its marker from a single address set within the Theme Options. It's meant to be used for one main "Contact Us" page. The client's business has several locations, and she wants each location's page to include such a map. (Example linked below.) It seems that the ideal solution would be the ability to specify an address within a shortcode in the Post, and have that set the map marker. Here's how the theme makes the map (where $mapAddress is the address from the Theme Options): <?php $mapAddress = ot_get_option( 'map_address' ); $mapHeight = ot_get_option( 'map_height' ); $mapContent = ot_get_option( 'map_content' ); ?> <section id="block-map-wrapper"> <div id="block-map" class="clearfix"> <script type="text/javascript" src="https://maps.googleapis.com/maps/api/js?sensor=true"></script> <script> jQuery(document).ready(function(){ // Map Options var mapOptions = { zoom: 15, scrollwheel: false, zoomControl: true, zoomControlOptions: { style: google.maps.ZoomControlStyle.SMALL, position: google.maps.ControlPosition.TOP_LEFT }, mapTypeControl: true, scaleControl: false, panControl: false, mapTypeId: google.maps.MapTypeId.ROADMAP }; // The Map Object var map = new google.maps.Map(document.getElementById("map"), mapOptions); var address = ""; var geocoder = new google.maps.Geocoder(); geocoder.geocode({ "address" : "<?php echo $mapAddress; ?>" }, function (results, status) { if (status == google.maps.GeocoderStatus.OK) { address = results[0].geometry.location; map.setCenter(results[0].geometry.location); var marker = new google.maps.Marker({ position: address, map: map, clickable: true, animation: google.maps.Animation.DROP }); var infowindow = new google.maps.InfoWindow({ content: "<?php echo $mapContent; ?>" }); google.maps.event.addListener(marker, "click", function() { infowindow.open(map, marker); }); } }); }); </script> <div id="map" class = "map" style = "width: 100%; height: <?php echo $mapHeight; ?>px"></div> </div><!-- #block-map --> <div class="shadow-bottom"></div> </section><!-- #block-map-wrapper --> Here's a test page using a custom Page template I've created. Right now it's using the same map code as above. I've tried creating a shortcode that takes an address attribute and sets it as $mapAddress, but that didn't work. I believe it's because the map is already loaded by the time the Loop gets parsed. How can I tell Maps to "come back" to the post to get the proper address? My specialty lies in HTML & CSS, but Javascript befuddles me fairly easily, so please be explicit when explaining implementation. Bonus: A further goal is to have the locations' parent Page also include such a map, but have multiple markers representing the multiple locations. When taking more than one location, Google Maps only accepts latitude/longitude. I don't want my client to be concerned with coordinates, so I know there's got to be something I can do with the geocoding service so she can just input a list of addresses instead (into the same, or similar, shortcode solution developed for my main question). But I am extra-clueless about how to do that.

    Read the article

  • Capturing wildcards in java generics

    - by Rollerball
    From this orcale java tutorial: The WildcardError example produces a capture error when compiled: import java.util.List; public class WildcardError { void foo(List<?> i) { i.set(0, i.get(0)); } } After this error demonstration, they fix the problem by using a helper method: public class WildcardFixed { void foo(List<?> i) { fooHelper(i); } // Helper method created so that the wildcard can be captured // through type inference. private <T> void fooHelper(List<T> l) { l.set(0, l.get(0)); } } First, they say that the list input parameter (i) is seen as an Object: In this example, the compiler processes the i input parameter as being of type Object. Why then i.get(0) does not return an Object? if it was already passed in as such? Furthermore what is the point of using a <?> when then you have to use an helper method using <T>. Would not be better using directly which can be inferred?

    Read the article

  • showing error on uploading a big file using php

    - by user1489969
    I have created a php code to display the upload option to upload multiple files as below: <?php $f_id= $_GET["id"]; ?> <title>Upload File</title> <form enctype="multipart/form-data" method="post" action="upload_hal_mult.php?id=<?php echo $f_id;?>" > <input type="hidden" name="MAX_FILE_SIZE" value="10000000"> <input id="infile" type="file" name="infile[]" multiple="true" /> <input type="submit" value="upload" name="file_uploaded" / > <br> <br> </form> So this will call "upload_hal_mult.php" when "upload" button is clicked. And the code for that is as follows: <title>Upload Results</title> <?php define("MAX_SIZE",10000000); $f_id= $_GET["id"]; $dir_name="dir_hal_".$f_id; $u=0; if (!is_dir($dir_name)) mkdir($dir_name); $dir=$dir_name."/"; $file_realname = $_FILES['infile']['name']; for ($i = 0; $i < count($_FILES['infile']['name']); $i++) { $ext = substr(strrchr($_FILES['infile']['name'][$i], "."), 1); $fname = substr($_FILES['infile']['name'][$i],0,strpos($_FILES['infile']['name'][$i], ".")); $fPath = $fname."_(".substr(md5(rand() * time()),0,4).")".".$ext"; echo "files size=".$_FILES["infile"]["size"][$i]."\n"; if($_FILES["infile"]["size"][$i]>MAX_SIZE) echo('File uploaded exceeds maximum upload size.'); if(($_FILES['infile']['error'][i]==0) && move_uploaded_file($_FILES['infile']['tmp_name'][$i], $dir . $fPath)) { $u=$u+1; ?> <!--<script type="text/javascript">setTimeout("window.close();", 1300);</script>--> <?php echo "Upload is successful\n"; } else echo "if stmt failed so error \n"; } if($u!=count($_FILES['infile']['name'])) echo "Error"; else echo "count is correct"; ?> This upload works correctly for files of size<10MB. But for files of size10MB, it's not echoing 'File uploaded exceeds maximum upload size.' as expected. Its also not uploading the file of size10MB. But the $u gets incremented. But none of the statements like "Upload is successful" or "if stmt failed so error" are being echoed as well. However the statement "count is correct" is being displayed, and this shows the $u got incremented somehow even though the echo statements didnt work! Can someone please point out the error I am doing here? I thought its simply a matter of 'if/else' statements but it seems more than that to me. Please help me out if you have any clue. Thanks

    Read the article

  • LINQ extention SelectMany in 3.5 vs 4.0?

    - by Moberg
    Hi When I saw Darins suggestion here .. IEnumerable<Process> processes = new[] { "process1", "process2" } .SelectMany(Process.GetProcessesByName); ( http://stackoverflow.com/questions/3059667/process-getprocessesbyname/3059733#3059733 ) .. I was a bit intrigued and I tried it in VS2008 with .NET 3.5 - and it did not compiling unless I changed it to .. IEnumerable<Process> res = new string[] { "notepad", "firefox", "outlook" } .SelectMany(s => Process.GetProcessesByName(s)); Having read some Darins answers before I suspected that it was me that were the problem, and when I later got my hands on a VS2010 with.NET 4.0 - as expected - the original suggestion worked beautifully. My question is : What have happend from 3.5 to 4.0 that makes this (new syntax) possible? Is it the extentionmethods that have been extended(hmm) or new rules for lambda syntax or? I've tried to search but my google-fu was not strong enough. Please forgive if the question is a bit naive and note that I've taged it as beginner :)

    Read the article

  • Scalability 101: How can I design a scalable web application using PHP?

    - by Legend
    I am building a web-application and have a couple of quick questions. From what I learnt, one should not worry about scalability when initially building the app and should only start worrying when the traffic increases. However, this being my first web-application, I am not quite sure if I should take an approach where I design things in an ad-hoc manner and later "fix" them. I have been reading stories about how people start off with an app that gets millions of users in a week or two. Not that I will face the same situation but I can't help but wonder, how do these people do it? Currently, I bought a shared hosting account on Lunarpages and that got me started in building and testing the application. However, I am interested in learning how to build the same application in a scalable-manner using the cloud, for instance, Amazon's EC2. From my understanding, I can see a couple of components: There is a load balancer that first receives requests and then decides where to route each request This request is then handled by a server replica that then processes the request and updates (if required) the database and sends back the response to the client If a similar request comes in, then a caching mechanism like memcached kicks into picture and returns objects from the cache A blackbox that handles database replication Specifically, I am trying to do the following: Setting up a load balancer (my homework revealed that HAProxy is one such load balancer) Setting up replication so that databases can be synchronized Using memcached Configuring Apache to work with multiple web servers Partitioning application to use Amazon EC2 and Amazon S3 (my application is something that will need great deal of storage) Finally, how can I avoid burning myself when using Amazon services? Because this is just a learning phase, I can probably do with 2-3 servers with a simple load balancer and replication but until I want to avoid paying loads of money accidentally. I am able to find resources on individual topics but am unable to find something that starts off from the big picture. Can someone please help me get started?

    Read the article

  • reporting tool/viewer for large datasets

    - by FrustratedWithFormsDesigner
    I have a data processing system that generates very large reports on the data it processes. By "large" I mean that a "small" execution of this system produces about 30 MB of reporting data when dumped into a CSV file and a large dataset is about 130-150 MB (I'm sure someone out there has a bigger idea of "large" but that's not the point... ;) Excel has the ideal interface for the report consumers in the form of its Data Lists: users can filter and segment the data on-the-fly to see the specific details that they are interested in - they can also add notes and markup to the reports, create charts, graphs, etc... They know how to do all this and it's much easier to let them do it if we just give them the data. Excel was great for the small test datasets, but it cannot handle these large ones. Does anyone know of a tool that can provide a similar interface as Excel data lists, but that can handle much larger files? The next tool I tried was MS Access, and found that the Access file bloats hugely (30 MB input file leads to about 70 MB Access file, and when I open the file, run a report and close it the file's at 120-150 MB!), the import process is slow and very manual (currently, the CSV files are created by the same plsql script that runs the main process so there's next to no intervention on my part). I also tried an Access database with linked tables to the database tables that store the report data and that was many times slower (for some reason, sqlplus could query and generate the report file in a minute or soe while Access would take anywhere from 2-5 minutes for the same data) (If it helps, the data processing system is written in PL/SQL and runs on Oracle 10g.)

    Read the article

  • can this problem be solved with a single SQL query?

    - by PierrOz
    I have the two following tables (with some sample datas) LOGS: ID | SETID | DATE ======================== 1 | 1 | 2010-02-25 2 | 2 | 2010-02-25 3 | 1 | 2010-02-26 4 | 2 | 2010-02-26 5 | 1 | 2010-02-27 6 | 2 | 2010-02-27 7 | 1 | 2010-02-28 8 | 2 | 2010-02-28 9 | 1 | 2010-03-01 STATS: ID | OBJECTID | FREQUENCY | STARTID | ENDID ============================================= 1 | 1 | 0.5 | 1 | 5 2 | 2 | 0.6 | 1 | 5 3 | 3 | 0.02 | 1 | 5 4 | 4 | 0.6 | 2 | 6 5 | 5 | 0.6 | 2 | 6 6 | 6 | 0.4 | 2 | 6 7 | 1 | 0.35 | 3 | 7 8 | 2 | 0.6 | 3 | 7 9 | 3 | 0.03 | 3 | 7 10 | 4 | 0.6 | 4 | 8 11 | 5 | 0.6 | 4 | 8 7 | 1 | 0.45 | 5 | 9 8 | 2 | 0.6 | 5 | 9 9 | 3 | 0.02 | 5 | 9 Every day new logs are analyzed on different sets of objects and stored in table LOGS. Among other processes, some statistics are computed on the objects contained into these sets and the result are stored in table STATS. These statistic are computed through several logs (identified by the STARTID and ENDID columns). So, what could be the SQL query that would give me the latest computed stats for all the objects with the corresponding log dates. In the given example, the result rows would be: OBJECTID | SETID | FREQUENCY | STARTDATE | ENDDATE ====================================================== 1 | 1 | 0.45 | 2010-02-27 | 2010-03-01 2 | 1 | 0.6 | 2010-02-27 | 2010-03-01 3 | 1 | 0.02 | 2010-02-27 | 2010-03-01 4 | 2 | 0.6 | 2010-02-26 | 2010-02-28 5 | 2 | 0.6 | 2010-02-26 | 2010-02-28 So, the most recent stats for set 1 are computed with logs from feb 27 to march 1 whereas stats for set 2 are computed from feb 26 to feb 28. object 6 is not in the results rows as there is no stat on it within the last period of time. Last thing, I use MySQL. Any Idea ?

    Read the article

  • Parallel doseq for Clojure

    - by andrew cooke
    I haven't used multithreading in Clojure at all so am unsure where to start. I have a doseq whose body can run in parallel. What I'd like is for there always to be 3 threads running (leaving 1 core free) that evaluate the body in parallel until the range is exhausted. There's no shared state, nothing complicated - the equivalent of Python's multiprocessing would be just fine. So something like: (dopar 3 [i (range 100)] ; repeated 100 times in 3 parallel threads... ...) Where should I start looking? Is there a command for this? A standard package? A good reference? So far I have found pmap, and could use that (how do I restrict to 3 at a time? looks like it uses 32 at a time - no, source says 2 + number of processors), but it seems like this is a basic primitive that should already exist somewhere. clarification: I really would like to control the number of threads. I have processes that are long-running and use a fair amount of memory, so creating a large number and hoping things work out OK isn't a good approach (example which uses a significant chunk available mem). update: Starting to write a macro that does this, and I need a semaphore (or a mutex, or an atom i can wait on). Do semaphores exist in Clojure? Or should I use a ThreadPoolExecutor? It seems odd to have to pull so much in from Java - I thought parallel programming in Clojure was supposed to be easy... Maybe I am thinking about this completely the wrong way? Hmmm. Agents?

    Read the article

  • Update transaction in SQL Server 2008 R2 from ASP.Net not working

    - by Amarus
    Hello! Even though I've been a stalker here for ages, this is the first post I'm making. Hopefully, it won't end here and more optimistically future posts might actually be me trying to give a hand to someone else, I do owe this community that much and more. Now, what I'm trying to do is simple and most probably the reason behind it not working is my own stupidity. However, I'm stumped here. I'm working on an ASP.Net website that interacts with an SQL Server 2008 R2 database. So far everything has been going okay but updating a row (or more) just won't work. I even tried copying and pasting code from this site and others but it's always the same thing. In short: No exception or errors are shown when the update command executes (it even gives the correct count of affected rows) but no changes are actually made on the database. Here's a simplified version of my code (the original had more commands and tons of parameters each, but even when it's like this it doesn't work): protected void btSubmit_Click(object sender, EventArgs e) { using (SqlConnection connection = new SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["ApplicationServices"].ConnectionString)) { string commandString = "UPDATE [impoundLotAlpha].[dbo].[Vehicle]" + "SET [VehicleMake] = @VehicleMake" + " WHERE [ComplaintID] = @ComplaintID"; using (SqlCommand command = new SqlCommand(commandString, connection)) { SqlTransaction transaction = null; try { command.Connection.Open(); transaction = connection.BeginTransaction(IsolationLevel.Serializable); command.Transaction = transaction; SqlParameter complaintID = new SqlParameter("@complaintID", SqlDbType.Int); complaintID.Value = HttpContext.Current.Request.QueryString["complaintID"]; command.Parameters.Add(complaintID); SqlParameter VehicleMake = new SqlParameter("@VehicleMake", SqlDbType.VarChar, 20); VehicleMake.Value = tbVehicleMake.Text; command.Parameters.Add(VehicleMake); command.ExecuteNonQuery(); transaction.Commit(); } catch { transaction.Rollback(); throw; } finally { connection.Close(); } } } } I've tried this with the "SqlTransaction" stuff and without it and nothing changes. Also, since I'm doing multiple updates at once, I want to have them act as a single transaction. I've found that it can be either done like this or by use of the classes included in the System.Transactions namespace (CommittableTransaction, TransactionScope...). I tried all I could find but didn't get any different results. The connection string in web.config is as follows: <connectionStrings> <add name="ApplicationServices" connectionString="Data Source=localhost;Initial Catalog=ImpoundLotAlpha;Integrated Security=True" providerName="System.Data.SqlClient"/> </connectionStrings> So, tldr; version: What is the mistake that I did with that record update attempt? (Figured it out, check below if you're having a similar issue.) What is the best method to gather multiple update commands as a single transaction? Thanks in advance for any kind of help and/or suggestions! Edit: It seems that I was lacking some sleep yesterday cause this time it only took me 5 minutes to figure out my mistake. Apparently the update was working properly but I failed to notice that the textbox values were being overwritten in Page_Load. For some reason I had this part commented: if (IsPostBack) return; The second part of the question still stands. But should I post this as an answer to my own question or keep it like this?

    Read the article

  • How can I efficiently manipulate 500k records in SQL Server 2005?

    - by cdeszaq
    I am getting a large text file of updated information from a customer that contains updates for 500,000 users. However, as I am processing this file, I often am running into SQL Server timeout errors. Here's the process I follow in my VB application that processes the data (in general): Delete all records from temporary table (to remove last month's data) (eg. DELETE * FROM tempTable) Rip text file into the temp table Fill in extra information into the temp table, such as their organization_id, their user_id, group_code, etc. Update the data in the real tables based on the data computed in the temp table The problem is that I often run commands like UPDATE tempTable SET user_id = (SELECT user_id FROM myUsers WHERE external_id = tempTable.external_id) and these commands frequently time out. I have tried bumping the timeouts up to as far as 10 minutes, but they still fail. Now, I realize that 500k rows is no small number of rows to manipulate, but I would think that a database purported to be able to handle millions and millions of rows should be able to cope with 500k pretty easily. Am I doing something wrong with how I am going about processing this data? Please help. Any and all suggestions welcome.

    Read the article

  • unbuffered I/O in Linux

    - by stuck
    I'm writing lots and lots of data that will not be read again for weeks - as my program runs the amount of free memory on the machine (displayed with 'free' or 'top') drops very quickly, the amount of memory my app uses does not increase - neither does the amount of memory used by other processes. This leads me to believe the memory is being consumed by the filesystems cache - since I do not intend to read this data for a long time I'm hoping to bypass the systems buffers, such that my data is written directly to disk. I dont have dreams of improving perf or being a super ninja, my hope is to give a hint to the filesystem that I'm not going to be coming back for this memory any time soon, so dont spend time optimizing for those cases. On Windows I've faced similar problems and fixed the problem using FILE_FLAG_NO_BUFFERING|FILE_FLAG_WRITE_THROUGH - the machines memory was not consumed by my app and the machine was more usable in general. I'm hoping to duplicate the improvements I've seen but on Linux. On Windows there is the restriction of writing in sector sized pieces, I'm happy with this restriction for the amount of gain I've measured. is there a similar way to do this in Linux?

    Read the article

  • C# threading solution for long queries

    - by Eddie
    Senerio We have an application that records incidents. An external database needs to be queried when an incident is approved by a supervisor. The queries to this external database are sometimes taking a while to run. This lag is experienced through the browser. Possible Solution I want to use threading to eliminate the simulated hang to the browser. I have used the Thread class before and heard about ThreadPool. But, I just found BackgroundWorker in this post. MSDN states: The BackgroundWorker class allows you to run an operation on a separate, dedicated thread. Time-consuming operations like downloads and database transactions can cause your user interface (UI) to seem as though it has stopped responding while they are running. When you want a responsive UI and you are faced with long delays associated with such operations, the BackgroundWorker class provides a convenient solution. Is BackgroundWorker the way to go when handling long running queries? What happens when 2 or more BackgroundWorker processes are ran simultaneously? Is it handled like a pool?

    Read the article

  • How to limit TCP writes to particular size and then block untlil the data is read

    - by ustulation
    {Qt 4.7.0 , VS 2010} I have a Server written in Qt and a 3rd party client executable. Qt based server uses QTcpServer and QTcpSocket facilities (non-blocking). Going through the articles on TCP I understand the following: the original implementation of TCP mentioned the negotiable window size to be a 16-bit value, thus maximum being 65535 bytes. But implementations often used the RFC window-scale-extension that allows the sliding window size to be scalable by bit-shifting to yield a maximum of 1 gigabyte. This is implementation defined. This could have resulted in majorly different window sizes on receiver and sender end as the server uses Qt facilities without hardcoding any window size limit. Client 1st asks for all information it can based on the previous messages from the server before handling the new (accumulating) incoming messages. So at some point Server receives a lot of messages each asking for data of several MB's. This the server processes and puts it into the sender buffer. Client however is unable to handle the messages at the same pace and it seems that client’s receiver buffer is far smaller (65535 bytes maybe) than sender’s transmit window size. The messages thus get accumulated at sender’s end until the sender’s buffer is full too after which the TCP writes on sender would block. This however does not happen as sender buffer is much larger. Hence this manifests as increase in memory consumption on the sender’s end. To prevent this from happening, I used Qt’s socket’s waitForBytesWritten() with timeout set to -1 for infinite waiting period. This as I see from the behaviour blocks the thread writing TCP data until the data has actually been sensed by the receiver’s window (which will happen when earlier messages have been processed by the client at application level). This has caused memory consumption at Server end to be almost negligible. is there a better alternative to this (in Qt) if i want to restrict the memory consumption at server end to say x MB's? Also please point out if any of my understandings is incorrect.

    Read the article

< Previous Page | 650 651 652 653 654 655 656 657 658 659 660 661  | Next Page >