Search Results

Search found 11052 results on 443 pages for 'linked tables'.

Page 419/443 | < Previous Page | 415 416 417 418 419 420 421 422 423 424 425 426  | Next Page >

• Creating a music catalog in C# and extracting first 30 seconds as soon as the first words are sung

  - by Rad

  I already read a question: Separation of singing voice from music. I don’t need this complex audio processing. I only need some detection mechanism that would detect that there is some voice/vocal playing while the music is playing (or not playing) I need to extract first 30 seconds when a vocalist starts singing along with full band music. See question 2 below. I want to create a music catalog using ASP.NET MVC 2 and Silverlight clients and C#.NET 4.0 programming language that would be front store. On the backend I would also like to create a desktop WPF/Windows application to create the music catalog from already existing music files, most of which have metadata in them ID3v1, ID3v2.3, ID3v2.4, iTunes MP4, WMA, Vorbis Comments and APE Tags etc. I would possibly like to create a web service that would allow catalog contributors to upload a zipped album and trigger metadata extraction of music data and extraction of music segments as described below. I would be happy if I achieve no. 1 below. Let's say I have 1000ths of songs in mp3 (or other formats) grouped in subfolders using some classification (Genre, Artists, Albums, Composers or other groupings). I want to create tables in DB that would organize songs so they can be searched based on different criteria (year, length, above classification or by song title, description etc) like what iTune store allows to their customers. I want to extract metadata from various formats (I will try to get songs in mp3 format, but there may be other popular formats) and allow music Catalog manager person to add missing data from either desktop or web applications. He or other contributors can upload zipped music via an HTML or Silverlight upload or WPF. Can anybody suggest open source libraries, articles, code snippets that can do that in an automatic way using .NET and possibly SQL Server DB? My main questions are these. This is an audio processing challenge. I want to extract 2 segments of music (questions 1 and 2): 1. How to extract a music segment: 1-2 seconds before a vocal starts singing and up to 30 seconds from that point in time and 2. Much more challenging is to find repeating segments (One would usually find or recognize the names of the songs and songs are usually known by these refrains. How would I go about creating a list of songs that go great together like what Genius from iTune does? Is there any characteristics of music that can be used to match songs? The goal is for people quickly scan and recognize songs i.e. associate melody, words with a title/album so they can make intelligent decisions like buying a song, create similar mood songs. Thanks, Rad

  Read the article

• javascript problems when generating html reports from within java

  - by posdef

  Hi, I have been working on a Java project in which the reports will be generated in HTML, so I am implementing methods for creating these reports. One important functionality is to be able to have as much info as possible in the tables, but still not clutter too much. In other words the details should be available if the user wishes to take a look at them but not necessarily visible by default. I have done some searching and testing and found an interesting template for hiding/showing content with the use of CSS and javascript, the problem is that when I try the resultant html page the scripts dont work. I am not sure if it's due a problem in Java or in the javascript itself. I have compared the html code that java produces to the source where I found the template, they seem to match pretty well. Below are bits of my java code that generates the javascript and the content, i would greatly appreciate if anyone can point out the possible reasons for this problem: //goes to head private void addShowHideScript() throws IOException{ StringBuilder sb = new StringBuilder(); sb.append("<script type=\"text/javascript\" language=\"JavaScript\">\n"); sb.append("<!--function HideContent(d) {\n"); sb.append("document.getElementById(d).style.display=\"none\";}\n"); sb.append("function ShowContent(d) {\n"); sb.append("document.getElementById(d).style.display=\"block\";}\n"); sb.append("function ReverseDisplay(d) {\n"); sb.append("if(document.getElementById(d).style.display==\"none\")\n"); sb.append("{ document.getElementById(d).style.display=\"block\"; }\n"); sb.append("else { document.getElementById(d).style.display=\"none\"; }\n}\n"); sb.append("//--></script>\n"); out.write(sb.toString()); out.newLine(); } // body private String linkShowHideContent(String pathname, String divname){ StringBuilder sb = new StringBuilder(); sb.append("<a href=\"javascript:ReverseDisplay('"); sb.append(divname); sb.append("')\">"); sb.append(pathname); sb.append("</a>"); return sb.toString(); } // content out.write(linkShowHideContent("hidden content", "ex")); out.write("<div id=\"ex\" style=\"display:block;\">"); out.write("<p>Content goes here.</p></div>");

  Read the article

• Default Database Collations PenTesting Env

  - by dominicdinada

  I am using Ubuntu 9.10 with XAMPP ( Lampp "MYSQL 5.1.45 PHPMYADMIN 3.3.1 PHP 5.3.2 ) What my problem is, is that I set up my testing env to debug my scripts locally and when I did so there arose a problem. This problem is that I used firefox's addon SQLinject ME to test for weakness' and upon doing so it caused mysql to change the default local collations; character sets dir /opt/lampp/share/mysql/charsets/ collation connection latin1_general_ci (Global value) latin1_swedish_ci collation database latin1_swedish_ci collation server latin1_swedish_ci I have searched for quite sometime in regards to a solution to this problem and have come up with searching for the db.opt file which stores this information without success. Upon not finding a solution I removed lampp with the "sudo rm -fR /opt" command and reinstall and the problem still persists. I have tried to change the collations manually and still come up with the database displaying latin1_swedish_ci as the default language. Why is this a problem?? Why is it a problem with mysql? Because the application I am testing and debugging locally is built on the CodeIgnitor with Smarty framework and since this combination of framework is built to detect the LOCALES, Rather what the database defaults are I keep getting errors saying no language file for swedish...... Of course I could get the swedish language file to work around this problem but I do not feel the need to make this work around a perminant solution as with time when I move on to projects I will run into simular problems every time that A; When importing database files, backups etc it will default to import such databases as the locale swedish. B; As time passes on I might completly forget of this error and will be back to square one. I have found this code in searches for a fix,which seems to alter the tables to a desired Collaion; $value) { mysql_query("ALTER TABLE $value COLLATE latin1_general_ci"); }} echo "The collation of your database has been successfully changed!"; ? Which is handy to switch collations in One Schema at a time however this is not a fix when a framework doesnt care that the said database is in one langugae. It tests for the Default of the entire server. Someone with any knowledge of a purge or fix to this I would greatly appricate the help. One more final note is that when I was testing I only figured to back up the applications DataBase and not the entire Schema of the install. No matter if I uninstall or reinstall the database still seems to carry these problems.

  Read the article

• Strange things appear on running the program

  - by FILIaS

  Hey! I'm fixing a program but I'm facing a problem and I cant really realize what's the wrong on the code. I would appreciate any help. I didnt post all the code...but i think with this part you can get an idea of it. With the following function enter() I wanna add user commands' datas to a list. eg. user give the command: "enter james bond 007 gun" 'james' is supposed to be the name, 'bond' the surname, 007 the amount and the rest is the description. I use strtok in order to 'cut' the command,then i put each name on a temp array. Then i call InsertSort in order to put the datas on a linked list but in alphabetical order depending on the surname that users give. I wanna keep the list on order and put each time the elements on the right position. /* struct for all the datas that user enters on file*/ typedef struct catalog { char short_name[50]; char surname[50]; signed int amount; char description[1000]; struct catalog *next; }catalog,*catalogPointer; catalogPointer current; catalogPointer head = NULL; void enter(void)//user command: enter <name> <surname> <amount> <description> { int n,j=2,k=0; char temp[1500]; char command[1500]; while (command[j]!=' ' && command[j]!='\0') { temp[k]=command[j]; j++; k++; } temp[k]='\0'; char *curToken = strtok(temp," "); printf("temp is:%s \n",temp); char short_name[50],surname[50],description[1000]; signed int amount; //short_name=(char *)malloc(sizeof (char *)); //surname=(char *)malloc(sizeof (char *)); //description=(char *)malloc(sizeof (char *)); //amount=(int *)malloc(sizeof (int *)); printf("\nWhat you entered for saving:\n"); for (n = 0; curToken !='\0'; ++n) { if (curToken) { strncpy(short_name, curToken, sizeof (char *)); / } printf("Short Name: %s \n",short_name); curToken = strtok(NULL," "); if (curToken) strncpy(surname, curToken, sizeof (char *)); / printf("SurName: %s \n",surname); curToken = strtok(NULL," "); if (curToken) { char *chk; amount = (int) strtol(curToken, &chk, 10); if (!isspace(*chk) && *chk != 0) fprintf(stderr,"Warning: expected integer value for amount, received %s instead\n",curToken); } printf("Amount: %d \n",amount); curToken = strtok(NULL,"\0"); if (curToken) { strncpy(description, curToken, sizeof (char *)); } printf("Description: %s \n",description); break; } if (findEntryExists(head, surname) != NULL) printf("\nAn entry for <%s %s> is already in the catalog!\nNew entry not entered.\n",short_name,surname); else { printf("\nTry to entry <%s %s %d %s> in the catalog list!\n",short_name,surname,amount,description); InsertSort(&head,short_name, surname, amount, description); printf("\n**Entry done!**\n"); } // Maintain the list in alphabetical order by surname. } /********Uses special case code for the head end********/ void SortedInsert(catalog** headRef, catalogPointer newNode,char short_name[],char surname[],signed int amount,char description[]) { strcpy(newNode->short_name, short_name); strcpy(newNode->surname, surname); newNode->amount=amount; strcpy(newNode->description, description); // Special case for the head end if (*headRef == NULL||(*headRef)->surname >= newNode->surname) { newNode->next = *headRef; *headRef = newNode; } else { // Locate the node before the point of insertion catalogPointer current = *headRef; catalogPointer temp=current->next; while ( temp!=NULL ) { if(strcmp(temp->surname,newNode->surname)<0 ) current = temp; } newNode->next = temp; temp = newNode; } } // Given a list, change it to be in sorted order (using SortedInsert()). void InsertSort(catalog** headRef,char short_name[],char surname[],signed int amount,char description[]) { catalogPointer result = NULL; // build the answer here catalogPointer current = *headRef; // iterate over the original list catalogPointer next; while (current!=NULL) { next = current->next; // tricky - note the next pointer before we change it SortedInsert(&result,current,short_name,surname,amount,description); current = next; } *headRef = result; } Running the program I get these strange things (garbage?)... Choose your selection: enter james bond 007 gun Your command is: enter james bond 007 gun temp is:james What you entered for saving: Short Name: james SurName: Amount: 0 Description: 0T?? Try to entry james 0 0T?? in the catalog list! Entry done! Also I'm facing a problem on how to use the 'malloc' on this program. Thanks in advance. . .

  Read the article

• Can't read excel file after creating it using File.WriteAllText() function

  - by Srikanth Mattihalli

  public void ExportDataSetToExcel(DataTable dt) { HttpResponse response = HttpContext.Current.Response; response.Clear(); response.Charset = "utf-8"; response.ContentEncoding = Encoding.GetEncoding("utf-8"); response.ContentType = "application/vnd.ms-excel"; Random Rand = new Random(); int iNum = Rand.Next(10000, 99999); string extension = ".xls"; string filenamepath = AppDomain.CurrentDomain.BaseDirectory + "graphs\\" + iNum + ".xls"; string file_path = "graphs/" + iNum + extension; response.AddHeader("Content-Disposition", "attachment;filename=\"" + iNum + "\""); string query = "insert into graphtable(graphtitle,graphpath,creategraph,year) VALUES('" + iNum.ToString() + "','" + file_path + "','" + true + "','" + DateTime.Now.Year.ToString() + "')"; try { int n = connect.UpdateDb(query); if (n > 0) { resultLabel.Text = "Merge Successfull"; } else { resultLabel.Text = " Merge Failed"; } resultLabel.Visible = true; } catch { } using (StringWriter sw = new StringWriter()) { using (HtmlTextWriter htw = new HtmlTextWriter(sw)) { // instantiate a datagrid DataGrid dg = new DataGrid(); dg.DataSource = dt; //ds.Tables[0]; dg.DataBind(); dg.RenderControl(htw); File.WriteAllText(filenamepath, sw.ToString()); // File.WriteAllText(filenamepath, sw.ToString(), Encoding.UTF8); response.Write(sw.ToString()); response.End(); } } } Hi all, I have created an excel sheet from datatable using above function. I want to read the excel sheet programatically using the below connectionstring. This string works fine for all other excel sheets but not for the one i created using the above function. I guess it is because of excel version problem. OleDbConnection conn= new OleDbConnection("Data Source='" + path +"';provider=Microsoft.Jet.OLEDB.4.0;Extended Properties=Excel 8.0;";); Can anyone suggest a way by which i can create an excel sheet such that it is readable again using above query. I cannot use Microsoft InterOp library as it is not supported by my host.

  Read the article

• MS Access 2003 - Option Group frame: can I add text boxes that are part of the frame instead of rad

  - by Justin

  Ok so this maybe a simple/silly question but I don't know so here goes: In access let's say I want to have a frame control, so I click the option group button and add it to the desgin surface. However, I am not wanting to use this as a option group with radio button selection, instead I would like to add text boxes instead the frame, so that when I reference the frame, it references every control instead of it, hence the text boxes, cbo boxes, etc.....just as it would if they were radio option selections. So can you do this? I want whatever controls I add inside the frame to be easily referenced (i.e. make all controls visible just by using frameExample.visible = true) so that I can build my own tab control groupings..... can this be done? Thanks! EDIT: What I am trying to accomplish is having a form that includes a collection of controls (input controls - cbo boxes, text boxes, etc), that serve as the Main record information. These are saved to a table via an INSERT statement on button_click because this form is unbound. Next I have 8 categories that are relative per each main record (and data that goes along with it). Each of these categories could have a sub form area and a button click that bring it's relative form into the sub form area. These sub forms would be unbound as well as I would just save data via SQL statement. So i know I could accomplish this by running the insert statement from the parent form, on the main collection control's data that would create the KeyID number, then run a SQL statement that would turn around and load that KeyID number right back onto the page in a hidden text box. Then when I click one of the sub forms and load its relative collection of controls, I could then save that data along with KeyID for each of these sub-forms/tables. SO...... I was wondering if instead you could define these controls as a collection so that you could hide and make visible all the ones you need on button clicks and avoid the need for additional forms (subs). I know that if a user enters data into a text box, and then somewhere along the way that box becomes hidden, the data still exists in it and still ends up in the SQL statement.... So I want all these controls to exist on the same form, but I thought what is I could encapsulate them into a frame like an option group, then I could call the frame and all the relative controls would be called up (made visible) as needed. Sorry for the long explanation but I thought it would help.

  Read the article

• How to designing a generic databse whos layout may change over time?

  - by mawg

  Here's a tricky one - how do I programatically create and interrogate a database who's contents I can't really foresee? I am implementing a generic input form system. The user can create PHP forms with a WYSIWYG layout and use them for any purpose he wishes. He can also query the input. So, we have three stages: a form is designed and generated. This is a one-off procedure, although the form can be edited later. This designs the database. someone or several people make use of the form - say for daily sales reports, stock keeping, payroll, etc. Their input to the forms is written to the database. others, maybe management, can query the database and generate reports. Since these forms are generic, I can't predict the database structure - other than to say that it will reflect HTML form fields and consist of a the data input from collection of edit boxes, memos, radio buttons and the like. Questions and remarks: A) how can I best structure the database, in terms of tables and columns? What about primary keys? My first thought was to use the control name to identify each column, then I realized that the user can edit the form and rename, so that maybe "name" becomes "employee" or "wages" becomes ":salary". I am leaning towards a unique number for each. B) how best to key the rows? I was thinking of a timestamp to allow me to query and a column for the row Id from A) C) I have to handle column rename/insert/delete. Foe deletion, I am unsure whether to delete the data from the database. Even if the user is not inputting it from the form any more he may wish to query what was previously entered. Or there may be some legal requirements to retain the data. Any gotchas in column rename/insert/delete? D) For the querying, I can have my PHP interrogate the database to get column names and generate a form with a list where each entry has a database column name, a checkbox to say if it should be used in the query and, based on column type, some selection criteria. That ought to be enough to build searches like "position = 'senior salesman' and salary 50k". E) I probably have to generate some fancy charts - graphs, histograms, pie charts, etc for query results of numerical data over time. I need to find some good FOSS PHP for this. F) What else have I forgotten? This all seems very tricky to me, but I am database n00b - maybe it is simple to you gurus?

  Read the article

• Rails Joins and include columns from joins table

  - by seth.vargo

  I don't understand how to get the columns I want from rails. I have two models - A User and a Profile. A User :has_many Profile (because users can revert back to an earlier version of their profile): > DESCRIBE users; +----------------+--------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +----------------+--------------+------+-----+---------+----------------+ | id | int(11) | NO | PRI | NULL | auto_increment | | username | varchar(255) | NO | UNI | NULL | | | password | varchar(255) | NO | | NULL | | | last_login | datetime | YES | | NULL | | +----------------+--------------+------+-----+---------+----------------+   > DESCRIBE profiles; +----------------+--------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +----------------+--------------+------+-----+---------+----------------+ | id | int(11) | NO | PRI | NULL | auto_increment | | user_id | int(11) | NO | MUL | NULL | | | first_name | varchar(255) | NO | | NULL | | | last_name | varchar(255) | NO | | NULL | | | . . . . . . | | . . . . . . | | . . . . . . | +----------------+--------------+------+-----+---------+----------------+ In SQL, I can run the query: > SELECT * FROM profiles JOIN users ON profiles.user_id = users.id LIMIT 1; +----+-----------+----------+---------------------+---------+---------------+-----+ | id | username | password | last_login | user_id | first_name | ... | +----+-----------+----------+---------------------+---------+---------------+-----+ | 1 | john | ****** | 2010-12-30 18:04:28 | 1 | John | ... | +----+-----------+----------+---------------------+---------+---------------+-----+ See how I get all the columns for BOTH tables JOINED together? However, when I run this same query in Rails, I don't get all the columns I want - I only get those from Profile: # in rails console >> p = Profile.joins(:user).limit(1) >> [#<Profile ...>] >> p.first_name >> NoMethodError: undefined method `first_name' for #<ActiveRecord::Relation:0x102b521d0> from /Library/Ruby/Gems/1.8/gems/activerecord-3.0.1/lib/active_record/relation.rb:373:in `method_missing' from (irb):8 # I do NOT want to do this (AKA I do NOT want to use "includes") >> p.user >> NoMethodError: undefined method `user' for #<ActiveRecord::Relation:0x102b521d0> from /Library/Ruby/Gems/1.8/gems/activerecord-3.0.1/lib/active_record/relation.rb:373:in method_missing' from (irb):9 I want to (efficiently) return an object that has all the properties of Profile and User together. I don't want to :include the user because it doesn't make sense. The user should always be part of the most recent profile as if they were fields within the Profile model. How do I accomplish this?

  Read the article

• Zend database query result converts column values to null

  - by David Zapata

  Hi again. I am using the next instructions to get some registers from my Database. Create the needed models (from the params module): $obj_paramtype_model = new Params_Model_DbTable_Paramtype(); $obj_param_model = new Params_Model_DbTable_Param(); Getting the available locales from the database // This returns a Zend_Db_Table_Row_Abstract class object $obj_paramtype = $obj_paramtype_model->getParamtypeByValue('available_locales'); // This is a query used to add conditions to the next sentence. This is executed from the Params_Model_DbTable_Param instance class, that depends from Params_Model_DbTable_Paramtype class (reference map and dependentTables arrays are fine in both classes) $obj_select = $this->select()->where('deleted_at IS NULL')->order('name'); // Execute the next query, applying the select restrictions. This returns a Zend_Db_Table_Rowset_Abstract class object. This means "Find Params by Paramtype" $obj_params_rowset = $obj_paramtype->findDependentRowset('Params_Model_DbTable_Param', 'Paramtype', $obj_paramtype); // Here the firebug log displays the queries.... Zend_Registry::get('log')->debug($obj_params_rowset); I have a profiler for all my DB executions from Zend. At this point the log and profiler objects (that includes Firebug writers), shows the executed SQL Queries, and the last line displays the resulting Zend_Db_Table_Rowset_Abstract class object. If I execute the SQL Queries in some MySQL Client, the results are as expected. But the Zend Firebug log writer displays as NULL the column values with latin characters (ñ). In other words, the external SQL client shows es_CO | Español de Colombia and en_US | English of United States but the Query results from Zend displays (and returns) es_CO | null and en_US | English of United States. I've deleted the ñ character from Español de Colombia and the query results are just fine in my Zend Log Firebug screen, and in the final Zend Form element. The MySQL database, tables and columns are in UTF-8 - utf8_unicode_ci collation. All my zend framework pages are in UTF-8 charset. I'm using XAMPP 1.7.1 (PHP 5.2.9, Apache at port 90 and MySQL 5.1.33-community) running on Windows 7 Ultimate; Zend Framework 1.10.1. I'm sorry if there is so much information, but I don't really know why could that happen, so I tryed to provide as much related information as I could to help to find some answer.

  Read the article

• linux thread synchronization

  - by johnnycrash

  I am new to linux and linux threads. I have spent some time googling to try to understand the differences between all the functions available for thread synchronization. I still have some questions. I have found all of these different types of synchronizations, each with a number of functions for locking, unlocking, testing the lock, etc. gcc atomic operations futexes mutexes spinlocks seqlocks rculocks conditions semaphores My current (but probably flawed) understanding is this: semaphores are process wide, involve the filesystem (virtually I assume), and are probably the slowest. Futexes might be the base locking mechanism used by mutexes, spinlocks, seqlocks, and rculocks. Futexes might be faster than the locking mechanisms that are based on them. Spinlocks dont block and thus avoid context swtiches. However they avoid the context switch at the expense of consuming all the cycles on a CPU until the lock is released (spinning). They should only should be used on multi processor systems for obvious reasons. Never sleep in a spinlock. The seq lock just tells you when you finished your work if a writer changed the data the work was based on. You have to go back and repeat the work in this case. Atomic operations are the fastest synch call, and probably are used in all the above locking mechanisms. You do not want to use atomic operations on all the fields in your shared data. You want to use a lock (mutex, futex, spin, seq, rcu) or a single atomic opertation on a lock flag when you are accessing multiple data fields. My questions go like this: Am I right so far with my assumptions? Does anyone know the cpu cycle cost of the various options? I am adding parallelism to the app so we can get better wall time response at the expense of running fewer app instances per box. Performances is the utmost consideration. I don't want to consume cpu with context switching, spinning, or lots of extra cpu cycles to read and write shared memory. I am absolutely concerned with number of cpu cycles consumed. Which (if any) of the locks prevent interruption of a thread by the scheduler or interrupt...or am I just an idiot and all synchonization mechanisms do this. What kinds of interruption are prevented? Can I block all threads or threads just on the locking thread's CPU? This question stems from my fear of interrupting a thread holding a lock for a very commonly used function. I expect that the scheduler might schedule any number of other workers who will likely run into this function and then block because it was locked. A lot of context switching would be wasted until the thread with the lock gets rescheduled and finishes. I can re-write this function to minimize lock time, but still it is so commonly called I would like to use a lock that prevents interruption...across all processors. I am writing user code...so I get software interrupts, not hardware ones...right? I should stay away from any functions (spin/seq locks) that have the word "irq" in them. Which locks are for writing kernel or driver code and which are meant for user mode? Does anyone think using an atomic operation to have multiple threads move through a linked list is nuts? I am thinking to atomicly change the current item pointer to the next item in the list. If the attempt works, then the thread can safely use the data the current item pointed to before it was moved. Other threads would now be moved along the list. futexes? Any reason to use them instead of mutexes? Is there a better way than using a condition to sleep a thread when there is no work? When using gcc atomic ops, specifically the test_and_set, can I get a performance increase by doing a non atomic test first and then using test_and_set to confirm? *I know this will be case specific, so here is the case. There is a large collection of work items, say thousands. Each work item has a flag that is initialized to 0. When a thread has exclusive access to the work item, the flag will be one. There will be lots of worker threads. Any time a thread is looking for work, they can non atomicly test for 1. If they read a 1, we know for certain that the work is unavailable. If they read a zero, they need to perform the atomic test_and_set to confirm. So if the atomic test_and_set is 500 cpu cycles because it is disabling pipelining, causes cpu's to communicate and L2 caches to flush/fill .... and a simple test is 1 cycle .... then as long as I had a better ratio of 500 to 1 when it came to stumbling upon already completed work items....this would be a win.* I hope to use mutexes or spinlocks to sparilngly protect sections of code that I want only one thread on the SYSTEM (not jsut the CPU) to access at a time. I hope to sparingly use gcc atomic ops to select work and minimize use of mutexes and spinlocks. For instance: a flag in a work item can be checked to see if a thread has worked it (0=no, 1=yes or in progress). A simple test_and_set tells the thread if it has work or needs to move on. I hope to use conditions to wake up threads when there is work. Thanks!

  Read the article

• DOM memory issue with IE8 (inserting lots of JSON data)

  - by okie.floyd

  i am developing a small web-utility that displays some data from some database tables. i have the utility running fine on FF, Safari, Chrome..., but the memory management on IE8 is horrendous. the largest JSON request I do will return information to create around 5,000 or so rows in a table within the browser (3 columns in the table). i'm using jquery to get the data (via getJSON). to remove the old/existing table, i'm just doing a $('#my_table_tbody').empty(). to add the new info to the table, within the getJSON callback, i am just appending each table row that i am creating to a variable, and then once i have them all, i am using $('#my_table_tbody').append(myVar) to add it to the existing tbody. i don't add the table rows as they are created because that seems to be a lot slower than just adding them all at once. does anyone have any recommendation on what someone should do who is trying to add thousands of rows of data to the DOM? i would like to stay away from pagination, but i'm wondering if i don't have a choice. Update 1 So here is the code I was trying after the innerHTML suggestion: /* Assuming a div called 'main_area' holds the table */ document.getElementById('main_area').innerHTML = ''; $.getJSON("my_server", {my: JSON, args: are, in: here}, function(j) { var mylength = j.length; var k =0; var tmpText = ''; tmpText += /* Add the table, thead stuff, and tbody tags here */; for (k = mylength - 1; k = 0; k--) { /* stack overflow wont let me type greater than & less than signs here, so just assume that they are there. */ tmpText += 'tr class="' + j[k].row_class . '" td class="col1_class" ' + j[k].col1 + ' /td td class="col2_class" ' + j[k].col2 + ' /td td class="col3_class" ' + j[k].col3 + ' /td /tr'; } document.getElementById('main_area').innerHTML = tmpText; } That is the gist of it. I've also tried using just a $.get request, and having the server send the formatted HTML, and just setting that in the innerHTML (i.e. document.getElementById('main_area').innerHTML = j;). thanks for all of the replies. i'm floored with the fact that you all are willing to help.

  Read the article

• IDs necessary in update script not being stored (or even seen!?) (PHP MySQL)

  - by Derek

  Hi guys, I really need help with this one...have spent 3 hours trying to figure it out... Basically, I have 3 tables necessary for this function to work (the query and PHP)... Authors, Books and Users. An author can have many books, and a user can have many books - that's it. When the admin user selects to update a book, they are presented with a form, displaying the current data within the fields, very straight forward... However there is one tricky part, the admin user can change the author for a book (incase they make a mistake) and also change the user for which the book is associated with. When I select to update the single book information I am not getting any values what so ever for author_id or user_id. Meaning that when the user updates the book info, the associations with the user and author is being scrapped altogether (when before there was an association)... I cannot see why this is happening because I can clearly see the IDs for the users and authors for my option values (this is because they are in select dropdowns). Here is what my sql to retrieve the user ID is: SELECT user_id, name FROM users and then i have my select options which brings up all the users in the system: <label>This book belongs to:</label> <select name="name" id="name"> <option value="<?php echo $row['user_id']?>" SELECTED><?php echo $row['name']?> - Current</option> <?php while($row = mysql_fetch_array($result)) { ?> <option value="<?php echo $row['user_id']; if (isset($_POST['user_id']));?>"><?php echo $row['name']?></option> <?php } ?> In the presented HTML form, I can select the users (by name) and within the source code I can see the IDs (for the value) matching against the names of the users. Finally, in my script that performs the update, I have this: $book_id = $_POST['book_id']; $bookname = $_POST['bookname']; $booklevel = $_POST['booklevel']; $author_id = $_POST['author_id']; $user_id = $_POST['user_id']; $sql = "UPDATE books SET bookname= '".$bookname."', booklevel= '".$booklevel."', author_id='".$author_id."', user_id= '".$user_id."' WHERE book_id = ".$book_id; The result of this query returns no value for either author_id or user_id... Obviously in this question I have given the information for the user stuff (with the HTML being displayed) but im guessing that I have the same problem with authors aswell... How can I get these ID's passed to the script so that the change can be acknowledge!! :(

  Read the article

• ASP.NET MVC & EF4 Entity Framework - Are there any performance concerns in using the entities vs retrieving only the fields i need?

  - by Ant

  Lets say we have 3 tables, Users, Products, Purchases. There is a view that needs to display the purchases made by a user. I could lookup the data required by doing: from p in DBSet<Purchases>.Include("User").Include("Product") select p; However, I am concern that this may have a performance impact because it will retrieve the full objects. Alternatively, I could select only the fields i need: from p in DBSet<Purchases>.Include("User").Include("Product") select new SimplePurchaseInfo() { UserName = p.User.name, Userid = p.User.Id, ProductName = p.Product.Name ... etc }; So my question is: Whats the best practice in doing this? == EDIT Thanks for all the replies. [QUESTION 1]: I want to know whether all views should work with flat ViewModels with very specific data for that view, or should the ViewModels contain the entity objects. Real example: User reviews Products var query = from dr in productRepository.FindAllReviews() where dr.User.UserId = 'userid' select dr; string sql = ((ObjectQuery)query).ToTraceString(); SELECT [Extent1].[ProductId] AS [ProductId], [Extent1].[Comment] AS [Comment], [Extent1].[CreatedTime] AS [CreatedTime], [Extent1].[Id] AS [Id], [Extent1].[Rating] AS [Rating], [Extent1].[UserId] AS [UserId], [Extent3].[CreatedTime] AS [CreatedTime1], [Extent3].[CreatorId] AS [CreatorId], [Extent3].[Description] AS [Description], [Extent3].[Id] AS [Id1], [Extent3].[Name] AS [Name], [Extent3].[Price] AS [Price], [Extent3].[Rating] AS [Rating1], [Extent3].[ShopId] AS [ShopId], [Extent3].[Thumbnail] AS [Thumbnail], [Extent3].[Creator_UserId] AS [Creator_UserId], [Extent4].[Comment] AS [Comment1], [Extent4].[DateCreated] AS [DateCreated], [Extent4].[DateLastActivity] AS [DateLastActivity], [Extent4].[DateLastLogin] AS [DateLastLogin], [Extent4].[DateLastPasswordChange] AS [DateLastPasswordChange], [Extent4].[Email] AS [Email], [Extent4].[Enabled] AS [Enabled], [Extent4].[PasswordHash] AS [PasswordHash], [Extent4].[PasswordSalt] AS [PasswordSalt], [Extent4].[ScreenName] AS [ScreenName], [Extent4].[Thumbnail] AS [Thumbnail1], [Extent4].[UserId] AS [UserId1], [Extent4].[UserName] AS [UserName] FROM [ProductReviews] AS [Extent1] INNER JOIN [Users] AS [Extent2] ON [Extent1].[UserId] = [Extent2].[UserId] LEFT OUTER JOIN [Products] AS [Extent3] ON [Extent1].[ProductId] = [Extent3].[Id] LEFT OUTER JOIN [Users] AS [Extent4] ON [Extent1].[UserId] = [Extent4].[UserId] WHERE N'615005822' = [Extent2].[UserId] or from d in productRepository.FindAllProducts() from dr in d.ProductReviews where dr.User.UserId == 'userid' orderby dr.CreatedTime select new ProductReviewInfo() { product = new SimpleProductInfo() { Id = d.Id, Name = d.Name, Thumbnail = d.Thumbnail, Rating = d.Rating }, Rating = dr.Rating, Comment = dr.Comment, UserId = dr.UserId, UserScreenName = dr.User.ScreenName, UserThumbnail = dr.User.Thumbnail, CreateTime = dr.CreatedTime }; SELECT [Extent1].[Id] AS [Id], [Extent1].[Name] AS [Name], [Extent1].[Thumbnail] AS [Thumbnail], [Extent1].[Rating] AS [Rating], [Extent2].[Rating] AS [Rating1], [Extent2].[Comment] AS [Comment], [Extent2].[UserId] AS [UserId], [Extent4].[ScreenName] AS [ScreenName], [Extent4].[Thumbnail] AS [Thumbnail1], [Extent2].[CreatedTime] AS [CreatedTime] FROM [Products] AS [Extent1] INNER JOIN [ProductReviews] AS [Extent2] ON [Extent1].[Id] = [Extent2].[ProductId] INNER JOIN [Users] AS [Extent3] ON [Extent2].[UserId] = [Extent3].[UserId] LEFT OUTER JOIN [Users] AS [Extent4] ON [Extent2].[UserId] = [Extent4].[UserId] WHERE N'userid' = [Extent3].[UserId] ORDER BY [Extent2].[CreatedTime] ASC [QUESTION 2]: Whats with the ugly outer joins?

  Read the article

• The question about the basics of LINQ to SQL

  - by Alex

  I just started learning LINQ to SQL, and so far I'm impressed with the easy of use and good performance. I used to think that when doing LINQ queries like from Customer in DB.Customers where Customer.Age > 30 select Customer LINQ gets all customers from the database ("SELECT * FROM Customers"), moves them to the Customers array and then makes a search in that Array using .NET methods. This is very inefficient, what if there are hundreds of thousands of customers in the database? Making such big SELECT queries would kill the web application. Now after experiencing how actually fast LINQ to SQL is, I start to suspect that when doing that query I just wrote, LINQ somehow converts it to a SQL Query string SELECT * FROM Customers WHERE Age > 30 And only when necessary it will run the query. So my question is: am I right? And when is the query actually run? The reason why I'm asking is not only because I want to understand how it works in order to build good optimized applications, but because I came across the following problem. I have 2 tables, one of them is Books, the other has information on how many books were sold on certain days. My goal is to select books that had at least 50 sales/day in past 10 days. It's done with this simple query: from Book in DB.Books where (from Sale in DB.Sales where Sale.SalesAmount >= 50 && Sale.DateOfSale >= DateTime.Now.AddDays(-10) select Sale.BookID).Contains(Book.ID) select Book The point is, I have to use the checking part in several queries and I decided to create an array with IDs of all popular books: var popularBooksIDs = from Sale in DB.Sales where Sale.SalesAmount >= 50 && Sale.DateOfSale >= DateTime.Now.AddDays(-10) select Sale.BookID; BUT when I try to do the query now: from Book in DB.Books where popularBooksIDs.Contains(Book.ID) select Book It doesn't work! That's why I think that we can't use thins kinds of shortcuts in LINQ to SQL queries, like we can't use them in real SQL. We have to create straightforward queries, am I right?

  Read the article

• Reliable and fast way to convert a zillion ODT files in PDF?

  - by Marco Mariani

  I need to pre-produce a million or two PDF files from a simple template (a few pages and tables) with embedded fonts. Usually, I would stay low level in a case like this, and compose everything with a library like ReportLab, but I joined late in the project. Currently, I have a template.odt and use markers in the content.xml files to fill with data from a DB. I can smoothly create the ODT files, they always look rigth. For the ODT to PDF conversion, I'm using openoffice in server mode (and PyODConverter w/ named pipe), but it's not very reliable: in a batch of documents, there is eventually a point after which all the processed files are converted into garbage (wrong fonts and letters sprawled all over the page). Problem is not predictably reproducible (does not depend on the data), happens in OOo 2.3 and 3.2, in Ubuntu, XP, Server 2003 and Windows 7. My Heisenbug detector is ticking. I tried to reduce the size of batches and restarting OOo after each one; still, a small percentage of the documents are messed up. Of course I'll write about this on the Ooo mailing lists, but in the meanwhile, I have a delivery and lost too much time already. Where do I go? Completely avoid the ODT format and go for another template system. Suggestions? Anything that takes a few seconds to run is way too slow. OOo takes around a second and it sums to 15 days of processing time. I had to write a program for clustering the jobs over several clients. Keep the format but go for another tool/program for the conversion. Which one? There are many apps in the shareware or commercial repositories for windows, but trying each one is a daunting task. Some are too slow, some cannot be run in batch without buying it first, some cannot work from command line, etc. Open source tools tend not to reinvent the wheel and often depend on openoffice. Converting to an intermediate .DOC format could help to avoid the OOo bug, but it would double the processing time and complicate a task that is already too hairy. Try to produce the PDFs twice and compare them, discarding the whole batch if there's something wrong. Although the documents look equal, I know of no way to compare the binary content. Restart OOo after processing each document. it would take a lot more time to produce them it would lower the percentage of the wrong files, and make it very hard to identify them. Go for ReportLab and recreate the pages programmatically. This is the approach I'm going to try in a few minutes. Learn to properly format bulleted lists Thanks a lot.

  Read the article

• How to add an XML parameter to a stored procedure in C#?

  - by salvationishere

  I am developing a C# web application in VS 2008 which interacts with my Adventureworks database in my SQL Server 2008. Now I am trying to add new records to one of the tables which has an XML column in it. How do I do this? This is the error I'm getting: System.Data.SqlClient.SqlException was caught Message="XML Validation: Text node is not allowed at this location, the type was defined with element only content or with simple content. Location: /" Source=".Net SqlClient Data Provider" ErrorCode=-2146232060 Class=16 LineNumber=22 Number=6909 Procedure="AppendDataC" Server="." State=1 StackTrace: at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection) at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj) at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj) at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString) at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async) at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result) at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe) at System.Data.SqlClient.SqlCommand.ExecuteNonQuery() at ADONET_namespace.ADONET_methods.AppendDataC(DataRow d, Hashtable ht) in C:\Documents and Settings\Admin\My Documents\Visual Studio 2008\Projects\AddFileToSQL\AddFileToSQL\ADONET methods.cs:line 212 InnerException: And this is a portion of my code in C#: try { SqlConnection conn2 = new SqlConnection(connString); SqlCommand cmd = conn2.CreateCommand(); cmd.CommandText = "dbo.AppendDataC"; cmd.CommandType = CommandType.StoredProcedure; cmd.Connection = conn2; ... sqlParam10.SqlDbType = SqlDbType.VarChar; SqlParameter sqlParam11 = cmd.Parameters.AddWithValue("@" + ht["@col11"], d[10]); sqlParam11.SqlDbType = SqlDbType.VarChar; SqlParameter sqlParam12 = cmd.Parameters.AddWithValue("@" + ht["@col12"], d[11]); sqlParam12.SqlDbType = SqlDbType.Xml; ... conn2.Open(); cmd.ExecuteNonQuery(); //This is the line it fails on and then jumps //to the Catch statement conn2.Close(); errorMsg = "The Person.Contact table was successfully updated!"; } catch (Exception ex) { Right now in my text input MDF file I have the XML parameter as: '<Products><id>3</id><id>6</id><id>15</id></Products>' Is this valid format for XML?

  Read the article

• How to properly set a datatable to an gridview in code with an ObjectDataSource?

  - by Xaisoft

  Hello, I have an ObjectDataSource with an ID of ObjectDataSource1 on a webpage. I also have a gridview in which I am binding the ObjectDataSource.ID to the GridView.DataSourceID. The problem I get is when text is changed in a textbox, the code calls BrokerageTransactions.GetAllWithDt which returns a DataTable. I want to set this datatable as the DataSource for the GridView, but it is telling me that I can't set the DataSouce and DataSourceId together. How can I fix this? Code is below. Also. Why can't you set a DataSourceID and a DataSource when using an ObjectDataSource? Thanks, X protected void BrokerageChange(Object sender, EventArgs e) { BrokerageTransactions brokerageaccountdetails = new BrokerageTransactions(); DataSet ds = BrokerageAccount.GetBrkID2(new Guid(Membership.GetUser().ProviderUserKey.ToString()), ddlBrokerageDetails.SelectedItem.Text.ToString()); foreach (DataRow dr in ds.Tables[0].Rows) { brokerageaccountdetails.BrokerageId = new Guid(dr["BrkrgId"].ToString()); } ddlBrokerageDetails.SelectedItem.Value = brokerageaccountdetails.BrokerageId.ToString(); if (txtTransactionsTo.Text != "" && txtTransactionsFrom.Text != "") ObjectDataSource1.FilterExpression = "convert(CreateDt,System.DateTime)>Convert('" + Convert.ToDateTime(txtTransactionsFrom.Text) + "',System.DateTime) and Convert(CreateDt,System.DateTime)<convert('" + Convert.ToDateTime(txtTransactionsTo.Text.ToString()) + "',System.DateTime)"; else if (txtTransactionsFrom.Text != "") ObjectDataSource1.FilterExpression = "convert(CreateDt,System.DateTime)>convert('" + Convert.ToDateTime(txtTransactionsFrom.Text) + "',System.DateTime)"; else if (txtTransactionsTo.Text != "") ObjectDataSource1.FilterExpression = "convert(CreateDt,System.DateTime) <convert('" + Convert.ToDateTime(txtTransactionsTo.Text.ToString()) + "',System.DateTime)"; else ObjectDataSource1.FilterExpression = " "; grvBrokerage.DataSourceID = ObjectDataSource1.ID; grvBrokerage.DataBind(); DateTime dtTransFrom = Convert.ToDateTime("1/1/1900"); DateTime dtTransTo = System.DateTime.Today; //TransactionsTo Box is Empty if ((txtTransactionsFrom.Text.Length > 2) && (txtTransactionsTo.Text.Length < 2)) { dtTransFrom = Convert.ToDateTime(txtTransactionsFrom.Text); dtTransTo = System.DateTime.Today; } //TransactionsFrom Box is Empty if ((txtTransactionsFrom.Text.Length < 2) && (txtTransactionsTo.Text.Length > 2)) { dtTransFrom = Convert.ToDateTime("1/1/1900"); dtTransTo = Convert.ToDateTime(txtTransactionsTo.Text); } //TransactionsFrom Box and TransactionsTo Box is Not Empty if ((txtTransactionsFrom.Text.Length > 2) && (txtTransactionsTo.Text.Length > 2)) { dtTransFrom = Convert.ToDateTime(txtTransactionsFrom.Text); dtTransTo = Convert.ToDateTime(txtTransactionsTo.Text); } // Fails Here grvBrokerage.DataSource = BrokerageTransactions.GetAllWithDt(brokerageaccountdetails.BrokerageId, dtTransFrom, dtTransTo); grvBrokerage.DataBind(); }

  Read the article

• Is there anything else I can do to optimize this MySQL query?

  - by Legend

  I have two tables, Table A with 700,000 entries and Table B with 600,000 entries. The structure is as follows: Table A: +-----------+---------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-----------+---------------------+------+-----+---------+----------------+ | id | bigint(20) unsigned | NO | PRI | NULL | auto_increment | | number | bigint(20) unsigned | YES | | NULL | | +-----------+---------------------+------+-----+---------+----------------+ Table B: +-------------+---------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+---------------------+------+-----+---------+----------------+ | id | bigint(20) unsigned | NO | PRI | NULL | auto_increment | | number_s | bigint(20) unsigned | YES | MUL | NULL | | | number_e | bigint(20) unsigned | YES | MUL | NULL | | | source | varchar(50) | YES | | NULL | | +-------------+---------------------+------+-----+---------+----------------+ I am trying to find if any of the values in Table A are present in Table B using the following code: $sql = "SELECT number from TableA"; $result = mysql_query($sql) or die(mysql_error()); while($row = mysql_fetch_assoc($result)) { $number = $row['number']; $sql = "SELECT source, count(source) FROM TableB WHERE number_s < $number AND number_e > $number GROUP BY source"; $re = mysql_query($sql) or die(mysql_error); while($ro = mysql_fetch_array($re)) { echo $number."\t".$ro[0]."\t".$ro[1]."\n"; } } I was hoping that the query would go fast but then for some reason, it isn't terrible fast. My explain on the select (with a particular value of "number") gives me the following: mysql> explain SELECT source, count(source) FROM TableB WHERE number_s < 1812194440 AND number_e > 1812194440 GROUP BY source; +----+-------------+------------+------+-------------------------+------+---------+------+--------+----------------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+------------+------+-------------------------+------+---------+------+--------+----------------------------------------------+ | 1 | SIMPLE | TableB | ALL | number_s,number_e | NULL | NULL | NULL | 696325 | Using where; Using temporary; Using filesort | +----+-------------+------------+------+-------------------------+------+---------+------+--------+----------------------------------------------+ 1 row in set (0.00 sec) Is there any optimization that I can squeeze out of this? I tried writing a stored procedure for the same task but it doesn't even seem to work in the first place... It doesn't give any syntax errors... I tried running it for a day and it was still running which felt odd. CREATE PROCEDURE Filter() Begin DECLARE number BIGINT UNSIGNED; DECLARE x INT; DECLARE done INT DEFAULT 0; DECLARE cur1 CURSOR FOR SELECT number FROM TableA; DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1; CREATE TEMPORARY TABLE IF NOT EXISTS Flags(number bigint unsigned, count int(11)); OPEN cur1; hist_loop: LOOP FETCH cur1 INTO number; SELECT count(*) from TableB WHERE number_s < number AND number_e > number INTO x; IF done = 1 THEN LEAVE hist_loop; END IF; IF x IS NOT NULL AND x>0 THEN INSERT INTO Flags(number, count) VALUES(number, x); END IF; END LOOP hist_loop; CLOSE cur1; END

  Read the article

• string categorization strategies

  - by Andrew Heath

  I'm the one-man dev team on a fledgling military history website. One aspect of the site is a catalog of ~1,200 individual battles, including the nations & formations (regiments, divisions, etc) which took part. The formation information (as well as the other battle info) was manually imported from a series of books by a 10-man volunteer team. The formations were listed in groups with varying formatting and abbreviation patterns. At the time I set up the data collection forms I couldn't think of a good way to process that data... and elected to store it all as strings in the MySQL database and sort it out later. Well, "later" - as it tends to happen - has arrived. :-) Each battle has 2+ records in the database - one for each nation that participated. Each record has a formations text string listing the formations present as the volunteer chose to add them. Some real examples: 39th Grenadier Rgmt, 26th Volksgrenadier Division 2nd Luftwaffe Field Division, 246th Infantry Division 247th Rifle Division, 255th Tank Brigade 2nd Luftwaffe Field Division, SS Cavalry Division 28th Tank Brigade, 158th Rifle Division, 135th Rifle Division, 81st Tank Brigade, 242nd Tank Brigade 78th Infantry Division 3rd Kure Special Naval Landing Force, Tulagi Seaplane Base personnel 1st Battalion 505th Infantry Regiment The ultimate goal is for each individual force to have an ID, so that its participation can be traced throughout the battle database. Formation hierarchy, such as the final item above 1st Battalion (of the) 505th Infantry Regiment also needs to be preserved. In that case, 1st Battalion and 505th Infantry Regiment would be split, but 1st Battalion would be flagged as belonging to the 505th. In database terms, I think I want to pull the formation field out of the current battle info table and create three new tables: FORMATION [id] [name] FORMATION_HIERARCHY [id] [parent] [child] FORMATION_BATTLE [f_id] [battle_id] It's simple to explain, but complicated to enact. What I'm looking for from the SO community is just some tips on how best to tackle this problem. Ideally there's some sort of method to solving this that I'm not aware of. However, as a last resort, I could always code a classification framework and call my volunteers back to sort through 2,500+ records...

  Read the article

• how to get latest entry from a table for an item and do arithmatic operation on it?

  - by I Like PHP

  i have below tables tbl_rcv_items st_id | item_id |stock_opening_qnty |stock_received_qnty |stock_rcvd_date 14 1 0 70 2010-05-18 15 16 0 100 2010-05-06 16 10 0 59 2010-05-20 17 14 0 34 2010-05-20 20 1 70 5 2010-05-12 tbl_issu_items issue_id refer_issue_id item_id item_qntt item_updated 51 1 1 5 2010-05-18 19:34:29 52 1 16 6 2010-05-18 19:34:29 53 1 10 7 2010-05-18 19:34:29 54 1 14 8 2010-05-18 19:34:29 75 7 1 12 2010-05-18 19:40:52 76 7 16 1 2010-05-18 19:40:52 77 7 10 1 2010-05-18 19:40:52 78 7 14 1 2010-05-18 19:40:52 79 8 1 3 2010-05-19 11:28:50 80 8 16 5 2010-05-19 11:28:50 81 8 10 6 2010-05-19 11:28:50 82 8 14 7 2010-05-19 11:28:51 87 10 1 2 2010-05-19 12:51:03 88 10 16 0 2010-05-19 12:51:03 89 10 10 0 2010-05-19 12:51:03 90 10 14 0 2010-05-19 12:51:03 91 14 1 1 2010-05-19 18:43:58 92 14 14 3 2010-05-19 18:43:58 tbl_item_detail item_id item_name 1 shirt 2 belt 10 ball pen 14 vim powder 16 pant NOW if i want total available quantity for each item till today using both table total available quantity for an item =stock_opening_qnty+stock_received_qnty(LATEST ENTRY FROM (tbl_rcv_item) for that item id according to stock_rcvd_date) - SUM(item_qntt) for eg: if i want to know the available quantity for item_id=1 till today(25-05-2010) then it shoud be 70+5(latest entry for item_id till 25/5/2010)-23( issued till 25/5/2010)=52 i write below query , SELECT tri.item_id, tid.item_name, (tri.stock_opening_qnty + tri.stock_received_qnty) AS totalRcvQntt, SUM( tii.item_qntt ) AS totalIsudQntt FROM tbl_rcv_items tri JOIN tbl_issu_items tii ON tii.item_id = tri.item_id JOIN tbl_item_detail tid ON tid.item_id=tri.item_id WHERE tri.stock_rcvd_date <= CURDATE() GROUP BY (tri.item_id) which results Array ( [0] => Array ( [item_id] => 1 [item_name] => shirt [totalRcvQntt] => 70 [totalIsudQntt] => 46 ) [1] => Array ( [item_id] => 10 [item_name] => ball pen [totalRcvQntt] => 59 [totalIsudQntt] => 16 ) [2] => Array ( [item_id] => 14 [item_name] => vim powder [totalRcvQntt] => 34 [totalIsudQntt] => 20 ) [3] => Array ( [item_id] => 16 [item_name] => pant [totalRcvQntt] => 100 [totalIsudQntt] => 17 ) ) in above result total isuse quantity for shirt(item_id=1) shoube be 23 whereas results reflects 46 bcoz there are two row regrading item_id=1 in tbl_rcv_items, i only need the latest one(means which stock_rcvd_date is less than tommorow) please tell me where i doing mistake?? or rewrite the best query. thanks a lot!

  Read the article

• Need help with joins in sqlalchemy

  - by Steve

  I'm new to Python, as well as SQL Alchemy, but not the underlying development and database concepts. I know what I want to do and how I'd do it manually, but I'm trying to learn how an ORM works. I have two tables, Images and Keywords. The Images table contains an id column that is its primary key, as well as some other metadata. The Keywords table contains only an id column (foreign key to Images) and a keyword column. I'm trying to properly declare this relationship using the declarative syntax, which I think I've done correctly. Base = declarative_base() class Keyword(Base): __tablename__ = 'Keywords' __table_args__ = {'mysql_engine' : 'InnoDB'} id = Column(Integer, ForeignKey('Images.id', ondelete='CASCADE'), primary_key=True) keyword = Column(String(32), primary_key=True) class Image(Base): __tablename__ = 'Images' __table_args__ = {'mysql_engine' : 'InnoDB'} id = Column(Integer, primary_key=True, autoincrement=True) name = Column(String(256), nullable=False) keywords = relationship(Keyword, backref='image') This represents a many-to-many relationship. One image can have many keywords, and one keyword can relate back to many images. I want to do a keyword search of my images. I've tried the following with no luck. Conceptually this would've been nice, but I understand why it doesn't work. image = session.query(Image).filter(Image.keywords.contains('boy')) I keep getting errors about no foreign key relationship, which seems clearly defined to me. I saw something about making sure I get the right 'join', and I'm using 'from sqlalchemy.orm import join', but still no luck. image = session.query(Image).select_from(join(Image, Keyword)).\ filter(Keyword.keyword == 'boy') I added the specific join clause to the query to help it along, though as I understand it, I shouldn't have to do this. image = session.query(Image).select_from(join(Image, Keyword, Image.id==Keyword.id)).filter(Keyword.keyword == 'boy') So finally I switched tactics and tried querying the keywords and then using the backreference. However, when I try to use the '.images' iterating over the result, I get an error that the 'image' property doesn't exist, even though I did declare it as a backref. result = session.query(Keyword).filter(Keyword.keyword == 'boy').all() I want to be able to query a unique set of image matches on a set of keywords. I just can't guess my way to the syntax, and I've spent days reading the SQL Alchemy documentation trying to piece this out myself. I would very much appreciate anyone who can point out what I'm missing.

  Read the article

• Prevent SQL Injection in Dynamic column names

  - by Mr Shoubs

  I can't get away without writing some dynamic sql conditions in a part of my system (using Postgres). My question is how best to avoid SQL Injection with the method I am currently using. EDIT (Reasoning): There are many of columns in a number of tables (a number which grows (only) and is maintained elsewhere). I need a method of allowing the user to decide which (predefined) column they want to query (and if necessary apply string functions to). The query itself is far too complex for the user to write themselves, nor do they have access to the db. There are 1000's of users with varying requirements and I need to remain as flexible as possible - I shouldn't have to revisit the code unless the main query needs to change - Also, there is no way of knowing what conditions the user will need to use. I have objects (received via web service) that generates a condition (the generation method is below - it isn't perfect yet) for some large sql queries. The _FieldName is user editable (parameter name was, but it didn't need to be) and I am worried it could be an attack vector. I put double quotes (see quoted identifier) around the field name in an attempt to sanitize the string, this way it can never be a key word. I could also look up the field name against a list of fields, but it would be difficult to maintain on a timely basis. Unfortunately the user must enter the condition criteria, I am sure there must be more I can add to the sanatize method? and does quoting the column name make it safe? (my limited testing seems to think so). an example built condition would be "AND upper(brandloaded.make) like 'O%' and upper(brandloaded.make) not like 'OTHERBRAND'" ... Any help or suggestions are appreciated. Public Function GetCondition() As String Dim sb As New Text.StringBuilder 'put quote around the table name in an attempt to prevent some sql injection 'http://www.postgresql.org/docs/8.2/static/sql-syntax-lexical.html sb.AppendFormat(" {0} ""{1}"" ", _LogicOperator.ToString, _FieldName) Select Case _ConditionOperator Case ConditionOperatorOptions.Equals sb.Append(" = ") ... End Select sb.AppendFormat(" {0} ", Me.UniqueParameterName) 'for parameter Return Me.Sanitize(sb) End Function Private Function Sanitize(ByVal sb As Text.StringBuilder) As String 'compare against a similar blacklist mentioned here: http://forums.asp.net/t/1254125.aspx sb.Replace(";", "") sb.Replace("'", "") sb.Replace("\", "") sb.Replace(Chr(8), "") Return sb.ToString End Function Public ReadOnly Property UniqueParameterName() As String Get Return String.Concat(":" _UniqueIdentifier) End Get End Property

  Read the article

• FluentNHibernate Many-To-One References where Foreign Key is not to Primary Key and column names are

  - by Todd Langdon

  I've been sitting here for an hour trying to figure this out... I've got 2 tables (abbreviated): CREATE TABLE TRUST ( TRUSTID NUMBER NOT NULL, ACCTNBR VARCHAR(25) NOT NULL ) CONSTRAINT TRUST_PK PRIMARY KEY (TRUSTID) CREATE TABLE ACCOUNTHISTORY ( ID NUMBER NOT NULL, ACCOUNTNUMBER VARCHAR(25) NOT NULL, TRANSAMT NUMBER(38,2) NOT NULL POSTINGDATE DATE NOT NULL ) CONSTRAINT ACCOUNTHISTORY_PK PRIMARY KEY (ID) I have 2 classes that essentially mirror these: public class Trust { public virtual int Id {get; set;} public virtual string AccountNumber { get; set; } } public class AccountHistory { public virtual int Id { get; set; } public virtual Trust Trust {get; set;} public virtual DateTime PostingDate { get; set; } public virtual decimal IncomeAmount { get; set; } } How do I do the many-to-one mapping in FluentNHibernate to get the AccountHistory to have a Trust? Specifically, since it is related on a different column than the Trust primary key of TRUSTID and the column it is referencing is also named differently (ACCTNBR vs. ACCOUNTNUMBER)???? Here's what I have so far - how do I do the References on the AccountHistoryMap to Trust??? public class TrustMap : ClassMap<Trust> { public TrustMap() { Table("TRUST"); Id(x => x.Id).Column("TRUSTID"); Map(x => x.AccountNumber).Column("ACCTNBR"); } } public class AccountHistoryMap : ClassMap<AccountHistory> { public AccountHistoryMap() { Table("TRUSTACCTGHISTORY"); Id (x=>x.Id).Column("ID"); References<Trust>(x => x.Trust).Column("ACCOUNTNUMBER").ForeignKey("ACCTNBR").Fetch.Join(); Map(x => x.PostingDate).Column("POSTINGDATE"); ); I've tried a few different variations of the above line but can't get anything to work - it pulls back AccountHistory data and a proxy for the Trust; however it says no Trust row with given identifier. This has to be something simple. Anyone? Thanks in advance.

  Read the article

• Execution Plan Optimization when where clause is removed then added back

  - by nmushov

  I have a stored procedure that uses a table valued function which executes in 9 seconds. If I alter the table valued function and remove the where clause, the stored procedure executes in 3 seconds. If I add the where clause back, the query still executes in 3 seconds. I took a look at the execution plans and it appears that after I remove the where clause, the execution plan includes parallelism and the scan count for 2 of my tables drops for 50000 and 65000 down to 5 and 3. After I add the where clause back, the optimized execution plan still runs unless I run DBCC FREEPROCCACHE. Questions 1. Why would SQL Server start using the optimized execution plan for both queries only when I first remove the where clause? Is there a way to force SQL Server to use this execution plan? Also, this is a paramaterized all-in-one query that uses the (Parameter is null or Parameter) in the where clause, which I believe is bad for performance. RETURNS TABLE AS RETURN ( SELECT TOP (@PageNumber * @PageSize) CASE WHEN @SortOrder = 'Expensive' THEN ROW_NUMBER() OVER (ORDER BY SellingPrice DESC) WHEN @SortOrder = 'Inexpensive' THEN ROW_NUMBER() OVER (ORDER BY SellingPrice ASC) WHEN @SortOrder = 'LowMiles' THEN ROW_NUMBER() OVER (ORDER BY Mileage ASC) WHEN @SortOrder = 'HighMiles' THEN ROW_NUMBER() OVER (ORDER BY Mileage DESC) WHEN @SortOrder = 'Closest' THEN ROW_NUMBER() OVER (ORDER BY P1.Distance ASC) WHEN @SortOrder = 'Newest' THEN ROW_NUMBER() OVER (ORDER BY [Year] DESC) WHEN @SortOrder = 'Oldest' THEN ROW_NUMBER() OVER (ORDER BY [Year] ASC) ELSE ROW_NUMBER() OVER (ORDER BY InventoryID ASC) END as rn, P1.InventoryID, P1.SellingPrice, P1.Distance, P1.Mileage, Count(*) OVER () RESULT_COUNT, dimCarStatus.[year] FROM (SELECT InventoryID, SellingPrice, Zip.Distance, Mileage, ColorKey, CarStatusKey, CarKey FROM facInventory JOIN @ZipCodes Zip ON Zip.DealerKey = facInventory.DealerKey) as P1 JOIN dimColor ON dimColor.ColorKey = P1.ColorKey JOIN dimCarStatus ON dimCarStatus.CarStatusKey = P1.CarStatusKey JOIN dimCar ON dimCar.CarKey = P1.CarKey WHERE (@ExteriorColor is NULL OR dimColor.ExteriorColor like @ExteriorColor) AND (@InteriorColor is NULL OR dimColor.InteriorColor like @InteriorColor) AND (@Condition is NULL OR dimCarStatus.Condition like @Condition) AND (@Year is NULL OR dimCarStatus.[Year] like @Year) AND (@Certified is NULL OR dimCarStatus.Certified like @Certified) AND (@Make is NULL OR dimCar.Make like @Make) AND (@ModelCategory is NULL OR dimCar.ModelCategory like @ModelCategory) AND (@Model is NULL OR dimCar.Model like @Model) AND (@Trim is NULL OR dimCar.Trim like @Trim) AND (@BodyType is NULL OR dimCar.BodyType like @BodyType) AND (@VehicleTypeCode is NULL OR dimCar.VehicleTypeCode like @VehicleTypeCode) AND (@MinPrice is NULL OR P1.SellingPrice >= @MinPrice) AND (@MaxPrice is NULL OR P1.SellingPrice < @MaxPrice) AND (@Mileage is NULL OR P1.Mileage < @Mileage) ORDER BY CASE WHEN @SortOrder = 'Expensive' THEN -SellingPrice WHEN @SortOrder = 'Inexpensive' THEN SellingPrice WHEN @SortOrder = 'LowMiles' THEN Mileage WHEN @SortOrder = 'HighMiles' THEN -Mileage WHEN @SortOrder = 'Closest' THEN P1.Distance WHEN @SortOrder = 'Newest' THEN -[YEAR] WHEN @SortOrder = 'Oldest' THEN [YEAR] ELSE InventoryID END )

  Read the article

• If Then Statement Condition Being Ignored With Optimisations On

  - by Matma

  I think im going mad but can some show me what im missing, it must be some stupidly simple i just cant see the wood for the trees. BOTH side of this if then else statement are being executed? Ive tried commenting out the true side and moving the condition to a seperate variable with the same result. However if i explicitly set the condition to 1=0 or 1=1 then the if then statement is operating as i would expect. i.e. only executing one side of the equation... The only time ive seen this sort of thing is when the compiler has crashed and is no longer compiling (without visible indication that its not) but ive restarted studio with the same results, ive cleaned the solution, built and rebuilt with no change? please show me the stupid mistake im making using vs2005 if it matters. Dim dset As DataSet = New DataSet If (CboCustomers.SelectedValue IsNot Nothing) AndAlso (CboCustomers.SelectedValue <> "") Then Dim Sql As String = "Select sal.SalesOrderNo As SalesOrder,cus.CustomerName,has.SerialNo, convert(varchar,sal.Dateofpurchase,103) as Date from [dbo].[Customer_Table] as cus " & _ " inner join [dbo].[Hasp_table] as has on has.CustomerID=cus.CustomerTag " & _ " inner join [dbo].[salesorder_table] as sal On sal.Hasp_ID =has.Hasp_ID Where cus.CustomerTag = '" & CboCustomers.SelectedValue.ToString & "'" Dim dap As SqlDataAdapter = New SqlDataAdapter(Sql, FormConnection) dap.Fill(dset, "dbo.Customer_Table") DGCustomer.DataSource = dset.Tables("dbo.Customer_Table") Else Dim erm As String = "wtf" End If EDIT: i have found that this is something to do with the release config settings im using, i guesing its the optimisations bit. does anyone know of any utils/addons for vs that show if a line has been optimised out. delphi, my former language showed blue dots in the left margin to show that it was a compiled line, no dot meaning it wasnt compiled in, is there anything like that for vs? alternatively can someone explain how optimisations would affect this simple if statement causeing it to run both sides? EDIT2: using this thread as possible causes/solutions : http://stackoverflow.com/questions/2135509/bug-only-occurring-when-compile-optimization-enabled. It does the same with release = optimisations on, x86, x64 and AnyCPU Goes away with optimisations off. Im using V2005 on a x64 win7 machine, if that matters. Thanks

  Read the article

< Previous Page | 415 416 417 418 419 420 421 422 423 424 425 426  | Next Page >