Search Results

Search found 11812 results on 473 pages for 'word processing'.

Page 396/473 | < Previous Page | 392 393 394 395 396 397 398 399 400 401 402 403  | Next Page >

• "User Friendly" .net compatible Regex/Text matching tools?

  - by Binary Worrier

  Currently in our software we provide a hook where we call a DLL built by our clients to parse information out of documents we are processing (the DLL takes in some text (or a file) and returns a list of name/value pairs). e.g. We're given a Word doc or Text file to Archive. We do various things to the file, and call a DLL that will return "pertinent" information about the file. Among other things we store that "pertinent" data for posterity. What is considered "pertinent" depends on the client and the type of the document, we don't care, we get it and store it. I've been asked to develop a user friendly "something" that will allow a non-programmer user to "configure" how to get this data from a plain text document (<humor>The user story ends with the helpful suggestion/query "We could use regex for this?"</humor>) It's safe to assume that a list of regex's isn't going to cut this, I've written some of these parsers for customers, the regex's to do these would be hedious and some of them can't be done by regex's. Also one of the requirements above is "user friendly" which negates anything that has users seeing or editing regex expressions. As you can guess, I don't have a fortune of time to do this, and am wondering is there anything out there that I can plug in to our app that has a nice front end and does exactly what I need? :) No? Whadda mean no! . . . sigh Ok then failing that, anything out there that "visually" builds regex's and/or other pattern matching expressions, and then allows one to run those expressions against some text? The MS BRE will do what I want, but I need something prettier that looks less like code. Thanks guys,

  Read the article

• Modify jQuery Highlight? Javascript Regex

  - by Matrym

  How can I modify jquery highlight such that it doesn't find matches that appear directly before or after an alpha character? In other words, how do I prevent a match mid-word? /* highlight v3 Highlights arbitrary terms. <http://johannburkard.de/blog/programming/javascript/highlight-javascript-text-higlighting-jquery-plugin.html> MIT license. Johann Burkard <http://johannburkard.de> <mailto:[email protected]> */ jQuery.fn.highlight = function(pat) { function innerHighlight(node, pat) { var skip = 0; if (node.nodeType == 3) { var pos = node.data.toUpperCase().indexOf(pat); if (pos >= 0) { var spannode = document.createElement('span'); spannode.className = 'highlight'; var middlebit = node.splitText(pos); var endbit = middlebit.splitText(pat.length); var middleclone = middlebit.cloneNode(true); spannode.appendChild(middleclone); middlebit.parentNode.replaceChild(spannode, middlebit); skip = 1; } } else if (node.nodeType == 1 && node.childNodes && !/(script|style)/i.test(node.tagName)) { for (var i = 0; i < node.childNodes.length; ++i) { i += innerHighlight(node.childNodes[i], pat); } } return skip; } return this.each(function() { innerHighlight(this, pat.toUpperCase()); }); }; jQuery.fn.removeHighlight = function() { return this.find("span.highlight").each(function() { this.parentNode.firstChild.nodeName; with (this.parentNode) { replaceChild(this.firstChild, this); normalize(); } }).end(); };

  Read the article

• Looking for Recommendations on Version Control System with Intuitive (.NET) API?

  - by John L.

  I'm working on a project which generates (composite) Microsoft Word documents which are comprised of one or more child documents. There are tens of thousands of permutations of the composite documents. Far too many for users to easily manage. Users will need to view/edit the child documents through the app which hides all of the nasty implementation details. A requirement of the system is that the child documents must be version controlled. That is what has been tripping me up. I've been torn between using an off-the-shelf solution or rolling my own. At a minimum, the system needs to support get latest, get specific version, add new, rename and possibly delete. I’ve whiteboarded it enough to realize it won’t be a trivial task to create my own. As far as commercial systems I have VSS and TFS at my disposal. I've played with the TFS API some, but it isn’t as intuitive or well documented as I had hoped. I'm not averse to an open source solution (e.g. SVN), but I have less familiarity with them. Which approach or tool would you recommend? Why? Do you have any links to API documentation you would recommend? Environment: C#, VS2008, SQL Server 2005/2008, low volume (a few hundred operations per day)

  Read the article

• What is the role/responsibility of a 'shell'?

  - by Rune

  Hi, I have been looking at the source code of the IronPython project and the Orchard CMS project. IronPython operates with a namespace called Microsoft.Scripting.Hosting.Shell (part of the DLR). The Orchard Project also operates with the concept of a 'shell' indirectly in various interfaces (IShellContainerFactory, IShellSettings). None of the projects mentioned above have elaborate documentation, so picking up the meaning of a type (class etc.) from its name is pretty valuable if you are trying to figure out the overall application structure/architecture by reading the source code. Now I am wondering: what do the authors of this source code have in mind when they refer to a 'shell'? When I hear the word 'shell', I think of something like a command line interpreter. This makes sense for IronPython, since it has an interactive interpreter. But to me, it doesn't make much sense with respect to a Web CMS. What should I think of, when I encounter something called a 'shell'? What is, in general terms, the role and responsibility of a 'shell'? Can that question even be answered? Is the meaning of 'shell' subjective (making the term useless)? Thanks.

  Read the article

• iPhone accelerometer:didAccelerate: seems to not get called while I am running a loop

  - by AJ

  An accelerometer related question. (Sorry the formatting may not look right, its the first time I am using this site). I got the accelerometer working as expected using the standard code UIAccelerometer *accel = [UIAccelerometer sharedAccelerometer]; accel.delegate = self; accel.updateInterval = 0.1; //I also tried other update values I use NSLog to log every time the accelerometer:didAccelerate: method in my class is called. The function gets called as expected and everything works fine till here. However, when I run a loop, the above method doesn't seem to get called. Something like this float firstAccelValue = globalAccel; //this is the x-accel value (stored in a global by the above method) float nextAccelValue = firstAccelValue; while (nextAccelValue == firstAccelValue){ //do something nextAccelValue = globalAccel; // note globalAccel is updated by the accelerometer method } The above loop never exits, expectedly since the accelerometer:didAccelerate: method is not getting called, and hence globalAccel never changes value. If I use a fixed condition to break the while loop, I can see that after the loop ends, the method calls work fine again. Am I missing something obvious here? Or does the accelerometer method not fire when certain processing is being done? Any help would be greatly appreciated! Thanks!

  Read the article

• How to use SQLAlchemy to dump an SQL file from query expressions to bulk-insert into a DBMS?

  - by Mahmoud Abdelkader

  Please bear with me as I explain the problem, how I tried to solve it, and my question on how to improve it is at the end. I have a 100,000 line csv file from an offline batch job and I needed to insert it into the database as its proper models. Ordinarily, if this is a fairly straight-forward load, this can be trivially loaded by just munging the CSV file to fit a schema, but I had to do some external processing that requires querying and it's just much more convenient to use SQLAlchemy to generate the data I want. The data I want here is 3 models that represent 3 pre-exiting tables in the database and each subsequent model depends on the previous model. For example: Model C --> Foreign Key --> Model B --> Foreign Key --> Model A So, the models must be inserted in the order A, B, and C. I came up with a producer/consumer approach: - instantiate a multiprocessing.Process which contains a threadpool of 50 persister threads that have a threadlocal connection to a database - read a line from the file using the csv DictReader - enqueue the dictionary to the process, where each thread creates the appropriate models by querying the right values and each thread persists the models in the appropriate order This was faster than a non-threaded read/persist but it is way slower than bulk-loading a file into the database. The job finished persisting after about 45 minutes. For fun, I decided to write it in SQL statements, it took 5 minutes. Writing the SQL statements took me a couple of hours, though. So my question is, could I have used a faster method to insert rows using SQLAlchemy? As I understand it, SQLAlchemy is not designed for bulk insert operations, so this is less than ideal. This follows to my question, is there a way to generate the SQL statements using SQLAlchemy, throw them in a file, and then just use a bulk-load into the database? I know about str(model_object) but it does not show the interpolated values. I would appreciate any guidance for how to do this faster. Thanks!

  Read the article

• How to read a database record with a DataReader and add it to a DataTable

  - by Olga

  Hello I have some data in a Oracle database table(around 4 million records) which i want to transform and store in a MSSQL database using ADO.NET. So far i used (for much smaller tables) a DataAdapter to read the data out of the Oracle DataBase and add the DataTable to a DataSet for further processing. When i tried this with my huge table, there was a outofmemory exception thrown. ( I assume this is because i cannot load the whole table into my memory) :) Now i am looking for a good way to perform this extract/transfer/load, without storing the whole table in the memory. I would like to use a DataReader and read the single dataRecords in a DataTable. If there are about 100k rows in it, I would like to process them and clear the DataTable afterwards(to have free memory again). Now i would like to know how to add a single datarecord as a row to a dataTable with ado.net and how to completly clear the dataTable out of memory: My code so far: Dim dt As New DataTable Dim count As Int32 count = 0 ' reads data records from oracle database table' While rdr.Read() 'read n records and add them to a dataTable' While count < 10000 dt.Rows.Add(????) count = count + 1 End While 'transform data in the dataTable, and insert it to the destination' ' flush the dataTable after insertion' count = 0 End While Thank you very much for your response!

  Read the article

• How do we greatly optimize our MySQL database (or replace it) when using joins?

  - by jkaz

  Hi there, This is the first time I'm approaching an extremely high-volume situation. This is an ad server based on MySQL. However, the query that is used incorporates a lot of JOINs and is generally just slow. (This is Rails ActiveRecord, btw) sel = Ads.find(:all, :select = '*', :joins = "JOIN campaigns ON ads.campaign_id = campaigns.id JOIN users ON campaigns.user_id = users.id LEFT JOIN countries ON countries.campaign_id = campaigns.id LEFT JOIN keywords ON keywords.campaign_id = campaigns.id", :conditions = [flashstr + "keywords.word = ? AND ads.format = ? AND campaigns.cenabled = 1 AND (countries.country IS NULL OR countries.country = ?) AND ads.enabled = 1 AND campaigns.dailyenabled = 1 AND users.uenabled = 1", kw, format, viewer['country'][0]], :order = order, :limit = limit) My questions: Is there an alternative database like MySQL that has JOIN support, but is much faster? (I know there's Postgre, still evaluating it.) Otherwise, would firing up a MySQL instance, loading a local database into memory and re-loading that every 5 minutes help? Otherwise, is there any way I could switch this entire operation to Redis or Cassandra, and somehow change the JOIN behavior to match the (non-JOIN-able) nature of NoSQL? Thank you!

  Read the article

• Send Email from worker role (Azure) with attachment in c#

  - by simplyvaibh

  I am trying to send an email(in c#) from worker role(Azure) with an attachment(from blob storage). I am able to send an email but attachment(word document) is blank. The following function is called from worker role. public void sendMail(string blobName) { InitStorage();//Initialize the storage var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString"); container = blobStorage.GetContainerReference("Container Name"); CloudBlockBlob blob = container.GetBlockBlobReference(blobName); if (File.Exists("demo.doc")) File.Delete("demo.doc"); FileStream fs = new FileStream("demo.doc", FileMode.OpenOrCreate); blob.DownloadToStream(fs); Attachment attach = new Attachment(fs,"Report.doc"); System.Net.Mail.MailMessage Email = new System.Net.Mail.MailMessage("[email protected]", "[email protected]"); Email.Subject = "Text fax send via email"; Email.Subject = "Subject Of email"; Email.Attachments.Add(attach); Email.Body = "Body of email"; System.Net.Mail.SmtpClient client = new SmtpClient("smtp.live.com", 25); client.DeliveryMethod = SmtpDeliveryMethod.Network; client.EnableSsl = true; client.Credentials = new NetworkCredential("[email protected]", Password); client.Send(Email); fs.Flush(); fs.Close(); Email.Dispose(); } Please tell me where I am doing wrong?

  Read the article

• Flex: FileReference and Image unhandled IOErrorEvent

  - by deux11

  The following code shows a button that allows you to select a file (should be an image) and display it into an image component. When I select an invalid image (e.g. a word document), I get the following error: "Error #2044: Unhandled IOErrorEvent:. text=Error #2124: Loaded file is an unknown type." I know I can pass a FileFilter to the FileReference:browse call, but that's beyond the point. My question is... I want to handle the IOErrorEvent myself, what event listener am I missing? private var file:FileReference = new FileReference(); private function onBrowse():void { file.browse(null); file.addEventListener(Event.SELECT, handleFileSelect); file.addEventListener(Event.COMPLETE, handleFileComplete); } private function handleFileSelect(event:Event):void { file.load(); } private function handleFileComplete(event:Event):void { myImage.source = file.data; } private function handleImageIoError(evt:IOErrorEvent):void { Alert.show("IOErrorEvent"); } <mx:Button click="onBrowse()" label="Browse"/> <mx:Image id="myImage" width="100" height="100" ioError="handleImageIoError(event)"/>

  Read the article

• Disabling redraw in WinForms app

  - by Ryan

  I'm working on a C#.Net application which has a somewhat annoying bug in it. The main window has a number of tabs, each of which has a grid on it. When switching from one tab to another, or selecting a different row in a grid, it does some background processing, and during this the menu flickers as it's redrawn (File, Help, etc menu items as well as window icon and title). I tried disabling the redraw on the window while switching tabs/rows (WM_SETREDRAW message) at first. In one case, it works perfectly. In the other, it solves the immediate bug (title/menu flicker), but between disabling the redraw and enabling it again, the window is "transparent" to mouse clicks - there's a small window (<1 sec) in which I can click and it will, say, highlight an icon on my desktop, as if the app wasn't there at all. If I have something else running in the background (Firefox, say) it will actually get focus when clicked (and draw part of the browser, say the address bar.) Here's code I added. m = new Message(); m.HWnd = System.Windows.Forms.Application.OpenForms[0].Handle; //top level m.WParam = (IntPtr)0; //disable redraw m.LParam = (IntPtr)0; //unused m.Msg = 11; //wm_setredraw WndProc(ref m); <snip - Application ignores clicks while in this section (in one case) m = new Message(); m.HWnd = System.Windows.Forms.Application.OpenForms[0].Handle; //top level m.WParam = (IntPtr)1; //enable m.LParam = (IntPtr)0; //unused m.Msg = 11; //wm_setredraw WndProc(ref m); System.Windows.Forms.Application.OpenForms[0].Refresh(); Does anyone know if a) there's a way to fix the transparent-application problem here, or b) if I'm doing it wrong in the first place and this should be fixed some other way?

  Read the article

• Parsing tab delimited file with double quotes in Perl

  - by sfactor

  I have a data set that is tab delimited with the user-agent strings in double quotes. I need to parse each of these columns and based on the answer of my other post I used the Text::CSV module. 94410634 0 GET "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; GTB6.6; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; AskTB5.5)" 1 The code is a simple one. #!/usr/bin/perl use strict; use warnings; use Text::CSV; my $csv = Text::CSV->new(sep_char => "\t"); while (<>) { if ($csv->parse($_)) { my @columns = $csv->fields(); print "@columns\n"; } else { my $err = $csv->error_input; print "Failed to parse line: $err"; } } But i get the Failed to parse line: error when I try it on this dataset. what am I doing wrong? I need to extract the 4th column containing the user-agent strings for further processing.

  Read the article

• jquery select elements between two elements that are not siblings

  - by naugtur

  eg. [I've removed attributes, but it's a bit of auto-generated html] <img class="p"/> <div> hello world <p> <font><font size="2">text.<img class="p"/> some text</font></font> </p> <img class="p"/> <p> <font><font size="2">more text<img class="p"/> another piece of text </font></font> </p><img class="p"/> some text on the end </div> I need to apply some highlighting with backgrounds to all text that is between two closest [in the html code] img.p elements when hovering first of them. I have no idea how to do that. Lets say I hover the first img.p - it should highlight hello world and text. and nothing else. And now the worst part - I need the backgrounds to disappear on mouseleave. I need it to work with any HTML mess possible. The above is just an example and structure of the documents will differ. [tip. Processing the whole html before binding hover and putting some spans etc. is ok as long as it doesn't change the looks of the output document]

  Read the article

• Using $.get with jquery validation

  - by Jimmy McCarthy

  I'm trying to use the Jquery builtin validator on my page. The issue is that I have certain fields that only are required if the JobID (entered into another field) does not already exist in our database. I have a simple service which simply takes JobID and returns True or False based on whether the JobID exists, but I can't seem to get this information where I want it. Some sample code: $("#dep_form").validate({ rules: { JobID: { required: true, digits: true, minlength: 3 }, OrgName: { required: function(element) { //This field is required if the JobID is new. return $("#jobinfo").html().length==15; } } }, messages: { JobID: { required: "Enter a Job Number.", digits: "Only numbers are allowed in Job ID's.", minlength: "Job Number must be at least 3 digits" }, OrgName: { required: "Select a Practice from the dropdown list." } }, errorClass: "ui-state-error-input", errorLabelContainer: "#errorbox", errorElement: 'li', errorContainer: "#validation_errors", onfocusout: false, onkeyup: false, focusinvalid: false }; Currently, I'm using a lazy method to validate (shown above). However, I now have access to a service using the URL: var lookupurl = "/deposits/jobidvalidate/?q=" + $("#id_JobID").val() + "&t=" + new Date().getTime(); which is a page which will contain just the word True or False based on whether that given JobID exists. I've tried half a dozen different ways of setting variables and calling functions within functions and still cannot get a way to simply return the value of that page (which I've been trying to access with $.get() ) to my validator, so that required is set to true when the Job does not exist and false if the job already exists. Any suggestions? Thanks.

  Read the article

• Matching a Repeating Sub Series using a Regular Expression with PowerShell

  - by Hinch

  I have a text file that lists the names of a large number of Excel spreadsheets, and the names of the files that are linked to from the spreadsheets. In simplified form it looks like this: "Parent File1.xls" Link: ChildFileA.xls Link: ChildFileB.xls "ParentFile2.xls" "ParentFile3.xls" Blah Link: ChildFileC.xls Link: ChildFileD.xls More Junk Link: ChildFileE.xls "Parent File4.xls" Link: ChildFileF.xls In this example, ParentFile1.xls has embedded links to ChildFileA.xls and ChildFileB.xls, ParentFile2.xls has no embedded links, and ParentFile3.xls has 3 embedded links. I am trying to write a regular expression in PowerShell that will parse the text file producing output in the following form: ParentFile1.xls:ChildFileA.xls,ChildFileB.xls ParentFile3.xls:ChildFileC.xls,ChildFileD.xls,ChildFileE.xls etc The task is complicated by the fact that the text file contains a lot of junk between each of the lines, and a parent may not always have a child. Furthermore, a single file name may pass over multiple lines. However, it's not as bad as it sounds, as the parent and child file names are always clearly demarcated (the parent with quotes and the child with a prefix of Link: ). The PowerShell code I've been using is as follows: $content = [string]::Join([environment]::NewLine, (Get-Content C:\Temp\text.txt)) $regex = [regex]'(?im)\s*\"(.*)\r?\n?\s*(.*)\"[\s\S]*?Link: (.*)\r?\n?' $regex.Matches($content) | %{$_.Groups[1].Value + $_.Groups[2].Value + ":" + $_.Groups[3].Value} Using the example above, it outputs: ParentFile1.xls:ChildFileA.xls ParentFile2.xls""ParentFile3.xls:ChildFileC.xls ParentFile4.xls:ChildFileF.xls There are two issues. Firstly, the inclusion of the "" instead of a newline whenever a Parent without a Child is processed. And the second issue, which is the most important, is that only a single child is ever shown for each parent. I'm guessing I need to somehow recursively capture and display the multiple child links that exist for each parent, but I'm totally stumped as to how to do this with a regular expression. Amy help would be greatly appreciated. The file contains 100's of thousands of lines, and manual processing is not an option :)

  Read the article

• I get an Exception when trying to implement reset password with Authlogic

  - by Tam

  Hi, I'm using Ruby on Rails 2.3.5 and Ruby 1.9 and implmeneted Authlogic as my authentication engine. Authlogic works fine. But now I have been trying to implement password reset using the following tutorial: http://www.binarylogic.com/2008/11/16/tutorial-reset-passwords-with-authlogic/ But I'm getting the following exception when I type mysite.com/password_resets/new Processing PasswordResetsController#new (for 127.0.0.1 at 2010-03-13 01:09:45) [GET] Completed in 9ms (DB: 0) | 200 [http://localhost/password_resets/new] [2010-03-13 01:09:45] ERROR TypeError: can't convert Array into String /Users/tammam56/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-2.3.5/lib/active_support/core_ext/string/output_safety.rb:34:in `concat' /Users/tammam56/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-2.3.5/lib/active_support/core_ext/string/output_safety.rb:34:in `concat_with_safety' /Users/tammam56/.rvm/gems/ruby-1.9.1-p378/gems/rack-1.0.1/lib/rack/handler/webrick.rb:63:in `block in service' /Users/tammam56/.rvm/gems/ruby-1.9.1-p378/gems/actionpack-2.3.5/lib/action_controller/response.rb:158:in `each' /Users/tammam56/.rvm/gems/ruby-1.9.1-p378/gems/actionpack-2.3.5/lib/action_controller/response.rb:158:in `each' /Users/tammam56/.rvm/gems/ruby-1.9.1-p378/gems/actionpack-2.3.5/lib/action_controller/string_coercion.rb:16:in `send' /Users/tammam56/.rvm/gems/ruby-1.9.1-p378/gems/actionpack-2.3.5/lib/action_controller/string_coercion.rb:16:in `method_missing' /Users/tammam56/.rvm/gems/ruby-1.9.1-p378/gems/actionpack-2.3.5/lib/action_controller/reloader.rb:22:in `method_missing' /Users/tammam56/.rvm/gems/ruby-1.9.1-p378/gems/rack-1.0.1/lib/rack/handler/webrick.rb:62:in `service' /Users/tammam56/.rvm/rubies/ruby-1.9.1-p378/lib/ruby/1.9.1/webrick/httpserver.rb:111:in `service' /Users/tammam56/.rvm/rubies/ruby-1.9.1-p378/lib/ruby/1.9.1/webrick/httpserver.rb:70:in `run' /Users/tammam56/.rvm/rubies/ruby-1.9.1-p378/lib/ruby/1.9.1/webrick/server.rb:183:in `block in start_thread' Honestly I'm not sure where to start debugging for this. Is it because I'm using Ruby 1.9? some people seem to be having trouble with Authlogic and Ruby 1.9: http://isitruby19.com/authlogic Please advise me how to go about solving/debugging this issues? Thanks, Tam

  Read the article

• What is the best way to determine the path to the ISV directory?

  - by Luke Baulch

  MSCRM 4.0 Problem: I'm currently storing xml files in the ISV directory along with my web applications. From a plugin (or potentially a seperate app), I need to find an easy way to navigate to the ISV directory to read these xml files. This routine will be called extremely often, so processing minimization should be a strong consideration. Potential solutions: Registry: There is a registry key called 'WebSitePath' with the data 'C:\Inetpub\wwwroot\CRM'. Could potentially use this to build the path. (Will this be the same on all systems/installations?) IIS directory data: Looping through the DirectoryEntries of path '"IIS://localhost/W3SVC"' I could obtain the the web application where description is equal to "Microsoft Dynamics CRM". (Will this be the same on all systems/installations?) Webservice: Create one to read and return the data contained in these xml files The webservice would have easy access to its executing directory. Database: Store the data of these files in the database. Help: Can anyone suggest a simpler solution to obtaining and reading a file from the ISV directory? If not, which of the above solutions would be the quickest to process? Thanks for any and all contributions.

  Read the article

• Fast JSON serialization (and comparison with Pickle) for cluster computing in Python?

  - by user248237

  I have a set of data points, each described by a dictionary. The processing of each data point is independent and I submit each one as a separate job to a cluster. Each data point has a unique name, and my cluster submission wrapper simply calls a script that takes a data point's name and a file describing all the data points. That script then accesses the data point from the file and performs the computation. Since each job has to load the set of all points only to retrieve the point to be run, I wanted to optimize this step by serializing the file describing the set of points into an easily retrievable format. I tried using JSONpickle, using the following method, to serialize a dictionary describing all the data points to file: def json_serialize(obj, filename, use_jsonpickle=True): f = open(filename, 'w') if use_jsonpickle: import jsonpickle json_obj = jsonpickle.encode(obj) f.write(json_obj) else: simplejson.dump(obj, f, indent=1) f.close() The dictionary contains very simple objects (lists, strings, floats, etc.) and has a total of 54,000 keys. The json file is ~20 Megabytes in size. It takes ~20 seconds to load this file into memory, which seems very slow to me. I switched to using pickle with the same exact object, and found that it generates a file that's about 7.8 megabytes in size, and can be loaded in ~1-2 seconds. This is a significant improvement, but it still seems like loading of a small object (less than 100,000 entries) should be faster. Aside from that, pickle is not human readable, which was the big advantage of JSON for me. Is there a way to use JSON to get similar or better speed ups? If not, do you have other ideas on structuring this? (Is the right solution to simply "slice" the file describing each event into a separate file and pass that on to the script that runs a data point in a cluster job? It seems like that could lead to a proliferation of files). thanks.

  Read the article

• Use $_FILES on a page called by .ajax

  - by RachelD

  I have two .php pages that I'm working with. Index.php has a file upload form that posts back to index.php. I can access the $_FILES no problem on index.php after submitting the form. My issue is that I want (after the form submit and the page loads) to use .ajax (jQuery) to call another .php file so that file can open and process some of the rows and return the results to ajax. The ajax then displays the results and recursively calls itself to process the next batch of rows. Basically I want to process (put in the DB etc) the csv in chunks and display it for the user in between chunks. Im doing it this way because the files are 400,000+ rows and the user doesnt want to wait the 10+ min for them all to be processed. I dont want to move this file (save it) because I just need to process it and throw it away and if a user closes the page while its processing the file wont be thrown away. I could cron script it but I dont want to. What I would really like to do is pass the (single) $_FILES through .ajax OR Save it in a $_POST or $_SESSION to use on the second page. Is there any hope for my cause? Heres the ajax code if that helps: function processCSV(startIndex, length) { $.ajax({ url: "ajax-targets/process-csv.php", dataType: "json", type: "POST", data: { startIndex: startIndex, length: length }, timeout: 60000, // 1000 = 1 sec success: function(data) { // JQuery to display the rows from the CSV var newStart = startIndex+length; if(newStart <= data['csvNumRows']) { processCSV(newStart, length); } } }); } processCSV(1, 2); }); P.S. I did try this Passing $_FILES or $_POST to a new page with PHP but its not working for me :( SOS.

  Read the article

• Strategies for serializing an object for auditing/logging purpose in .NET?

  - by Jiho Han

  Let's say I have an application that processes messages. Messages are just objects in this case that implements IMessage interface which is just a marker. In this app, if a message fails to process, then I want to log it, first of all for auditing and troubleshooting purposes. Secondly I might want to use it for re-processing. Ideally, I want the message to be serialized into a format that is human-readable. The first candidate is XML although there are others like JSON. If I were to serialize the messages as XML, I want to know whether the message object is XML-serializable. One way is to reflect on the type and to see if it has a parameter-less constructor and the other is to require IXmlSerializable interface. I'm not too happy with either of these approaches. There is a third option which is to try to serialize it and catch exceptions. This doesn't really help - I want to, in some way, stipulate that IMessage (or a derived type) should be xml-serializable. The reflection route has obvious disadvantages such as security, performance, etc. IXmlSerializable route locks down my messages to one format, when in the future, I might want to change the serialization format to be JSON. The other thing is even the simplest objects now must implement ReadXml and WriteXml methods. Is there a route that involves the least amount of work that lets me serialize an arbitrary object (as long as it implements the marker interface) into XML but not lock future messages into XML?

  Read the article

• Sudden issues reading uncompressed video using opencv

  - by JohnSavage

  I have been using a particular pipeline to process video using opencv to encode uncompressed video (fourcc = 0), and opencv python bindings to then open and work on these files. This has been working fine for me on OpenCV 2.3.1a on Ubuntu 11.10 until just a few days ago. For some reason it currently is only allowing me to read the first frame of a given file the first time I open that file. Further frames are not read, and once I touch the file once with my program, it then cannot even read the first frame. More detail: I created the uncompressed video files as follows: out_video.open(out_vid_name, 0, // FOURCC = 0 means record raw fps, Size(640, 480)) Again, these videos worked fine for me until about a week ago. Now, when I try to open one of these I get the following message (from what I think is ffmpeg): Processing video.avi Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later. [avi @ 0x29251e0] parser not found for codec rawvideo, packets or times may be invalid. It reads and displays the first frame fine, but then fails to read the next frame. Then, when I try to run my code on the same video, the capture still opens with the same message as above. However, it cannot even read the very first frame. Here is the code to open the capture: self.capture = cv2.VideoCapture(filename) if not self.capture.isOpened() print "Error: could not open capture" sys.exit() Again, this part is passed without any issue, but then the break happens at: success, rgb = self.capture.read() if not success: print "error: could not read frame" return False This part breaks at the second frame on the first run of the video file, and then on the first frame on subsequent runs. I really don't know where to even begin debugging this. Please help!

  Read the article

• suggestions for a 3D graph rendering library?

  - by Sandro

  Hello coders! So I'm not sure how stackoverflow friendly this question is since it doesn't have a quick clear cut answer but here we go... I have a java program that generates data for a directed graph. Now I need to render this graph. The data needs to be laid out in 3D, and I want to be able to define which plane an edge lives in. (Each edge will only need to occupy 1 plane of the 3D space). I also need the ability to navigate around the graph. Since I know that this kind of stuff is hard, I'm going shopping. So far I've looked into (In no particular order): JUNG: lacks 3D support Cytoscape: not sure how much I'll be able to define edge drawing, haven't seen a non bio-informatics application of it yet JGraph: I didn't see any 3D applications yet Perfuse: looks promising, does anyone know anything else about it? Gephi: Documentation looks scarce Processing: does this play well with java? I'm also considering doing some combination of opengl + swing rendering to create a 3D graph from multiple 2D graphs. I am also not adverse to the idea of linking from another language Any Ideas? Thank you.

  Read the article

• StorageClientException: The specified message does not exist?

  - by Aaron

  I have a simple video encoding worker role that pulls messages from a queue encodes a video then uploads the video to storage. Everything seems to be working but occasionally when deleting the message after I am done encoding and uploading I get a "StorageClientException: The specified message does not exist." Although the video is processed, I believe the message is reappearing in the queue because it's not being deleted correctly. Is it possible that another instance of the Worker role is processing and deleting the message? Doesn't the GetMessage() prevent other worker roles from picking up the same message? Am I doing something wrong in the setup of my queue? What could be causing this message to not be found on delete? some code... //onStart() queue setup var queueStorage = _storageAccount.CreateCloudQueueClient(); _queue = queueStorage.GetQueueReference(QueueReference); queueStorage.RetryPolicy = RetryPolicies.Retry(5, new TimeSpan(0, 5, 0)); _queue.CreateIfNotExist(); public override void Run() { while (true) { try { var msg = _queue.GetMessage(new TimeSpan(0, 5, 0)); if (msg != null) { EncodeIt(msg); PostIt(msg); _queue.DeleteMessage(msg); } else { Thread.Sleep(WaitTime); } } catch (StorageClientException exception) { BlobTrace.Write(exception.ToString()); Thread.Sleep(WaitTime); } } }

  Read the article

• How best to use XPath with very large XML files in .NET?

  - by glenatron

  I need to do some processing on fairly large XML files ( large here being potentially upwards of a gigabyte ) in C# including performing some complex xpath queries. The problem I have is that the standard way I would normally do this through the System.XML libraries likes to load the whole file into memory before it does anything with it, which can cause memory problems with files of this size. I don't need to be updating the files at all just reading them and querying the data contained in them. Some of the XPath queries are quite involved and go across several levels of parent-child type relationship - I'm not sure whether this will affect the ability to use a stream reader rather than loading the data into memory as a block. One way I can see of making it work is to perform the simple analysis using a stream-based approach and perhaps wrapping the XPath statements into XSLT transformations that I could run across the files afterward, although it seems a little convoluted. Alternately I know that there are some elements that the XPath queries will not run across, so I guess I could break the document up into a series of smaller fragments based on it's original tree structure, which could perhaps be small enough to process in memory without causing too much havoc. I've tried to explain my objective here so if I'm barking up totally the wrong tree in terms of general approach I'm sure you folks can set me right...

  Read the article

• Value is not changing in the javascript variable on the onClick event

  - by Chetan sharma

  I am trying to implement a dynamic search jquery and php, but I am not able to change the value of a javascript variable on my onclick event. <script type="text/javascript"> var main_category = '<?php echo $main_category?>'; var main_url = '<?php echo $url?>get/' + encodeURIComponent(main_category) + '/'; var container_id = '<?php echo $unique_id?>content_area'; //name $('#<?php echo $unique_id?>search_by_name').autocomplete('<?php echo $url?>get_autocomplete_data/' + encodeURIComponent(main_category) + '/names/', { matchContains: "word", autoFill: true, width: 310 });$('#<?php echo $unique_id?>search_by_name').result(function (event, data){ var url = main_url + 'search/title/' + encodeURIComponent(data); load_div(container_id, url); }); //on click of displayed categories $('[rel="<?php echo $unique_id?>sub_category"]').click(function (){ window['main_category'] = this.title; $('#<?php echo $unique_id?>category').html('' +main_category+ ''); return false; }); }); </script> It changes the value when on click is fired //on click of displayed categories $('[rel="<?php echo $unique_id?>sub_category"]').click(function (){ window['main_category'] = this.title; $('#<?php echo $unique_id?>category').html('' +main_category+ ''); return false; }); but when after that i add data for the search it still searches for the old category $('#<?php echo $unique_id?>search_by_name').result(function (event, data){ var url = main_url + 'search/title/' + encodeURIComponent(data); load_div(container_id, url); }); "main_category" value is not changing in the "main_url"

  Read the article

< Previous Page | 392 393 394 395 396 397 398 399 400 401 402 403  | Next Page >