Search Results

Search found 14017 results on 561 pages for 'mysql binlog'.

Page 536/561 | < Previous Page | 532 533 534 535 536 537 538 539 540 541 542 543  | Next Page >

• A Digg-like rotating homepage of popular content, how to include date as a factor?

  - by Ferdy

  I am building an advanced image sharing web application. As you may expect, users can upload images and others can comments on it, vote on it, and favorite it. These events will determine the popularity of the image, which I capture in a "karma" field. Now I want to create a Digg-like homepage system, showing the most popular images. It's easy, since I already have the weighted Karma score. I just sort on that descendingly to show the 20 most valued images. The part that is missing is time. I do not want extremely popular images to always be on the homepage. I guess an easy solution is to restrict the result set to the last 24 hours. However, I'm also thinking that in order to keep the image rotation occur throughout the day, time can be some kind of variable where its offset has an influence on the image's sorting. Specific questions: Would you recommend the easy scenario (just sort for best images within 24 hours) or the more sophisticated one (use datetime offset as part of the sorting)? If you advise the latter, any help on the mathematical solution to this? Would it be best to run a scheduled service to mark images for the homepage, or would you advise a direct query (I'm using MySQL) As an extra note, the homepage should support paging and on a quiet day should include entries of days before in order to make sure it is always "filled" I'm not asking the community to build this algorithm, just looking for some advise :)

  Read the article

• Implementing Model-level caching

  - by Byron

  I was posting some comments in a related question about MVC caching and some questions about actual implementation came up. How does one implement a Model-level cache that works transparently without the developer needing to manually cache, yet still remains efficient? I would keep my caching responsibilities firmly within the model. It is none of the controller's or view's business where the model is getting data. All they care about is that when data is requested, data is provided - this is how the MVC paradigm is supposed to work. (Source: Post by Jarrod) The reason I am skeptical is because caching should usually not be done unless there is a real need, and shouldn't be done for things like search results. So somehow the Model itself has to know whether or not the SELECT statement being issued to it worthy of being cached. Wouldn't the Model have to be astronomically smart, and/or store statistics of what is being most often queried over a long period of time in order to accurately make a decision? And wouldn't the overhead of all this make the caching useless anyway? Also, how would you uniquely identify a query from another query (or more accurately, a resultset from another resultset)? What about if you're using prepared statements, with only the parameters changing according to user input? Another poster said this: I would suggest using the md5 hash of your query combined with a serialized version of your input arguments. This would require twice the number of serialization options. I was under the impression that serialization was quite expensive, and for large inputs this might be even worse than just re-querying. And is the minuscule chance of collision worth worrying about? Conceptually, caching in the Model seems like a good idea to me, but it seems in practicality the developer should have direct control over caching and write it into the controller. Thoughts/ideas? Edit: I'm using PHP and MySQL if that helps to narrow your focus.

  Read the article

• I installed XAMPP in a virtual drive and now I can't run its services. Why?

  - by Haris

  Hi, The description is quite long. Please spend some time to read it. ^:)^ I have an old PHP application and I'm trying to test and debug it. Unfortunately, the application uses important data so I can't just click this and that. Now, what I'm trying to do is create a copy of the application in a different computer. From now on, I will call the computer running my original PHP application as 'Computer A' and the computer which I'm going to use to run the copy of the application as 'Computer B'. To prevent missing link problems since the application contains static paths, such in images or tags, I have to copy all files and folder related to my PHP application from Computer A to the same path in Computer B. Unfortunately, Computer B only has drive C while Computer A has drive D and the files of my PHP application is located in 'D:\xampp\htdocs' in Computer A. OK, now I have to create drive D in computer B. At first, I tried to create a second partition in Computer B by using PowerQuest Partition Magic 8, but somehow Partition Magic doesn't run in Computer B. I have tried to reinstall it but it still doesn't run. So, another alternative is to create a virtual drive. That is what I did. I created a virtual drive by running the 'subst' command in Command Prompt. The virtual drive is D and it refers to a directory, which is 'C:\Virtual'. After I have drive D in Computer B, I installed XAMPP there. The installation was successful. Now, I also have 'D:\xampp\htdocs' in Computer B. However, when I ran the Apache, MySQL, or Filezilla service, I receive an error message "Error 3: The system cannot find the file specified.". In Computer B, there is no IIS or process using the port 80. What should I do? Please help me. Many thanks in advance, Haris

  Read the article

• Bulk Insert of hundreds of millions of records

  - by Dave Jarvis

  What is the fastest way to insert 237 million records into a table that has rules (for distributing the data across 84 child tables)? First I tried inserts. No go. Then I tried inserts with BEGIN/COMMIT. Not nearly fast enough. Next, I tried COPY FROM, but then noticed the documentation states that the rules are ignored. (And it was having difficulties with the column order and date format -- it said that '1984-07-1' was not a valid integer; true, but a bit unexpected.) Some example data: station_id,taken,amount,category_id,flag 1,'1984-07-1',0,4, 1,'1984-07-2',0,4, 1,'1984-07-3',0,4, 1,'1984-07-4',0,4,T Here is the table structure (with one rule included): CREATE TABLE climate.measurement ( id bigserial NOT NULL, station_id integer NOT NULL, taken date NOT NULL, amount numeric(8,2) NOT NULL, category_id smallint NOT NULL, flag character varying(1) NOT NULL DEFAULT ' '::character varying ) WITH ( OIDS=FALSE ); ALTER TABLE climate.measurement OWNER TO postgres; CREATE OR REPLACE RULE i_measurement_01_001 AS ON INSERT TO climate.measurement WHERE date_part('month'::text, new.taken)::integer = 1 AND new.category_id = 1 DO INSTEAD INSERT INTO climate.measurement_01_001 (id, station_id, taken, amount, category_id, flag) VALUES (new.id, new.station_id, new.taken, new.amount, new.category_id, new.flag); I can generate the data into any format. Am looking for something that won't take four days. I originally had the data in MySQL (still do), but am hoping to get a performance increase by switching to PostgreSQL and am eager to use its PL/R extensions for stats. I was also thinking about using: http://pgbulkload.projects.postgresql.org/ Any help, tips, or guidance would be greatly appreciated. Thank you!

  Read the article

• How to estimate tomcat server requirements?

  - by Daniil

  We have a brand new webapp written that runs on Tomcat. So far, only one client is using it through the day. They run about 180 unique logins a day. Not really a lot IMO. Now, we managed to sell it to a brand new client who likes and wants to roll it out to 50,000 clients. How many of them will login at the same time - no idea. But I need to do the whole thing - allocate, create, config and maintain. OK - last is simple(errrr). The application runs off of Tomcat 5.5 on Gentoo (I'm thinking to upgrade to Tomcat 6) with MSSQL & mySQL behind. I do realize that a more enterprise ready application would be a better fit, but that's not an option at the moment. Since I've never done this before, I'm a bit lost. Can someone advice on how to go about estimating the equipment requirements for this client? Tomcat does have clustering, so that I can do. MS SQL - I'm sure they have something too. I'm thinking to stick it behind LVS (which we do use at the moment for something else too). Any help from people who deal with these details is greatly appreciated!

  Read the article

• How to query JDO persistent objects in unowned relationship model?

  - by Paul B

  Hello, I'm trying to migrate my app from PHP and RDBMS (MySQL) to Google App Engine and have a hard time figuring out data model and relationships in JDO. In my current app I use a lot of JOIN queries like: SELECT users.name, comments.comment FROM users, comments WHERE users.user_id = comments.user_id AND users.email = '[email protected]' As I understand, JOIN queries are not supported in this way so the only(?) way to store data is using unowned relationships and "foreign" keys. There is a documentation regarding that, but no useful examples. So far I have something like this: @PersistenceCapable public class Users {     @PrimaryKey     @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)     private Key key;     @Persistent     private String name;         @Persistent     private String email;         @Persistent     private Set<Key> commentKeys;     // Accessors... } @PersistenceCapable public class Comments {     @PrimaryKey     @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)     private Key key;     @Persistent     private String comment;         @Persistent     private Date commentDate;     @Persistent     private Key userKey;     // Accessors... } So, how do I get a list with commenter's name, comment and date in one query? I see how I probably could get away with 3 queries but that seems wrong and would create unnecessary overhead. Please, help me out with some code examples. -- Paul.

  Read the article

• Ajax long polling (comet) + php on lighttpd v1.4.22 multiple instances problem.

  - by fibonacci

  Hi, I am new to this site, so I really hope I will provide all the necessary information regarding my question. I've been trying to create a "new message arrived notification" using long polling. Currently I am initiating the polling request by window.onLoad event of each page in my site. On the server side I have an infinite loop: while(1){ if(NewMessageArrived($current_user))break; sleep(10); } echo $newMessageCount; On the client side I have the following (simplified) ajax functions: poll_new_messages(){ xmlhttp=GetXmlHttpObject(); //... xmlhttp.onreadystatechange=got_new_message_count; //... xmlhttp.send(); } got_new_message_count(){ if (xmlhttp.readyState==4){ updateMessageCount(xmlhttp.responseText); //... poll_new_messages(); } } The problem is that with each page load, the above loop starts again. The result is multiple infinite loops for each user that eventually make my server hang. *The NewMessageArived() function queries MySQL DB for new unread messages. *At the beginning of the php script I run start_session() in order to obtain the $current_user value. I am currently the only user of this site so it is easy for me to debug this behavior by writing time() to a file inside this loop. What I see is that the file is being written more often than once in 10 seconds, but it starts only when I go from page to page. Please let me know if any additional information might help. Thank you.

  Read the article

• Sanitizing user input that will later be e-mailed - what should I be worried about?

  - by Kevin Burke

  I'm interning for an NGO in India (Seva Mandir, http://sevamandir.org) and trying to fix their broken "subscribe to newsletter" box. Because the staff isn't very sophisticated and our web host isn't great, I decided to send the relevant data to the publications person via mail() instead of storing it in a MySQL database. I know that it's best to treat user input as malicious, and I've searched the SO forums for posts relevant to escaping user data for sending in a mail message. I know the data should be escaped; what should I be worried about and what's the best way to sanitize the input before emailing it? Form flow: 1. User enters email on homepage and clicks Submit 2. User enters name, address, more information on second page (bad usability, I know, but my boss asked me to) and clicks "Submit" 3. Collect the data via $_POST and email it to the publications editor (and possibly send a confirmation to the subscriber). I am going to sanitize the email in step 2 and the other data in step 3. I appreciate your help, Kevin

  Read the article

• Oracle why does creating trigger fail when there is a field called timestamp?

  - by Omar Kooheji

  I've just wasted the past two hours of my life trying to create a table with an auto incrementing primary key bases on this tutorial, The tutorial is great the issue I've been encountering is that the Create Target fails if I have a column which is a timestamp and a table that is called timestamp in the same table... Why doesn't oracle flag this as being an issue when I create the table? Here is the Sequence of commands I enter: Creating the Table: CREATE TABLE myTable (id NUMBER PRIMARY KEY, field1 TIMESTAMP(6), timeStamp NUMBER, ); Creating the Sequence: CREATE SEQUENCE test_sequence START WITH 1 INCREMENT BY 1; Creating the trigger: CREATE OR REPLACE TRIGGER test_trigger BEFORE INSERT ON myTable REFERENCING NEW AS NEW FOR EACH ROW BEGIN SELECT test_sequence.nextval INTO :NEW.ID FROM dual; END; / Here is the error message I get: ORA-06552: PL/SQL: Compilation unit analysis terminated ORA-06553: PLS-320: the declaration of the type of this expression is incomplete or malformed Any combination that does not have the two lines with a the word "timestamp" in them works fine. I would have thought the syntax would be enough to differentiate between the keyword and a column name. As I've said I don't understand why the table is created fine but oracle falls over when I try to create the trigger... CLARIFICATION I know that the issue is that there is a column called timestamp which may or may not be a keyword. MY issue is why it barfed when I tried to create a trigger and not when I created the table, I would have at least expected a warning. That said having used Oracle for a few hours, it seems a lot less verbose in it's error reporting, Maybe just because I'm using the express version though. If this is a bug in Oracle how would one who doesn't have a support contract go about reporting it? I'm just playing around with the express version because I have to migrate some code from MySQL to Oracle.

  Read the article

• Parsing xml with php and xpath

  - by Hyung Suh

  Hey guys, I'm trying to parse an xml file to return a item with a specific id only, but having trouble making it work. here's what I have in php $xml_str = file_get_contents("test.xml"); $xml = simplexml_load_string($xml_str); $albid = $_GET['id']; $nodes = $xml->xpath('//library/book[@id=1]'); foreach($nodes as $node) { echo $node['title'].'<br/>'; } First, the php is not returning anything from the xml file. What would I need to fix to return the data? Also, how would I enter $albid into the xpath so that the id will be retrieved from the link? Any pointers in the right direction would be appreciated. Thanks! --and here's the sample xml file-- <library> <book id="1"> <title>PHP and MySQL</title> <author fname="miguel" lname="alvarez">Miguel Alvarez</author> </book> <book id="2"> <title>JAVA 123</title> <author fname="william" lname="vega">WIlliam Vega</author> </book>

  Read the article

• Is there a way to prevent Maven Test from rebuilding the database?

  - by Christopher W. Allen-Poole

  I've recently been asked to, effectively, sell my department on unit testing. I can't tell you how excited this makes me, but I do have one concern. We're using JUnit with Spring and Maven, and this means that each time mvn test is called, it rebuilds the database. Obviously, we can't integrate that with our production server -- it would kill valuable data. How do I prevent the rebuilding without telling maven to skip testing? The best I could figure was to assign the script to operate in a test database (line breaks added for readability): mvn test -Ddbunit.schema=<database>test -Djdbc.url=jdbc:mysql://localhost/<database>test? createDatabaseIfNotExist=true&amp; useUnicode=true&amp;characterEncoding=utf-8 I can't help but think there must be a better way. I'm especially interested in learning if there is an easy way to tell Maven to only run tests on particular classes without building anything else? mvn -Dtest=<test-name> test still rebuilds the database. ======= update ======= Bit of egg on my face here. I didn't realize that I was using the same variable in two places, meaning that the POM was using a "skip.test" variable for both rebuilding the database and for running the tests...

  Read the article

• SQL Express Edition, SQL Compact Editin and SQLCMD for learning purpose

  - by Mil

  Hi, I want to learn programming in SQL from some SQL tutorial sites of which I heard of here but I need some environment for executing query's. I think I have both SQL CE and SQL EE installed on my computer but I have some doubts about these DBMS and I don't know exactly how to use SQLCMD utility so I hope someone here will have time and will to explain me the following: Since running sqlcmd -S.\sqlexpress at command prompt command gives "1" prompt I assume I have SQL express installed but anyway how can I be sure what I have installed on my machine since I cannot find in installed programs SQL Express Edition name? Can I ship and use database with my C# (VC# Express) application which was created with SQL EE (embedded?)? How can use sqlcmd for learning SQL, that is by issuing commands like create, use, select..., again emphasize is on learning SQL I do not want to run scripts but use interactive command prompt like with MySQL (since I want to use SQL I would pretty much like to avoid graphical tools for DBMS)? Please tell me if you have some other advice regarding as to what should I better use in learning how to program in SQL or should I stick with the above for now. Thanks in advance.

  Read the article

• How to sort a date array in PHP

  - by Click Upvote

  I have an array in this format: Array ( [0] => Array ( [28th February, 2009] => 'bla' ) [1] => Array ( [19th March, 2009] => 'bla' ) [2] => Array ( [5th April, 2009] => 'bla' ) [3] => Array ( [19th April, 2009] => 'bla' ) [4] => Array ( [2nd May, 2009] => 'bla' ) ) I want to sort them out in the ascending order of the dates (based on the month, day, and year). What's the best way to do that? Originally the emails are being fetched in the MySQL date format, so its possible for me to get the array in this state: Array [ ['2008-02-28']='some text', ['2008-03-06']='some text' ] Perhaps when its in this format, I can loop through them, remove all the '-' (hyphen) marks so they are left as integars, sort them using array_sort() and loop through them yet again to sort them? Would prefer if there was another way as I'd be doing 3 loops with this per user. Thanks. Edit: I could also do this: $array[$index]=array('human'=>'28 Feb, 2009', 'db'=>'20080228', 'description'=>'Some text here'); But using this, would there be any way to sort the array based on the 'db' element alone? Edit 2: Updated initial var_dump

  Read the article

• I need to get form data from multiple forms on one page using $_POST

  - by CDeanMartin

  My project is a menu that displays daily specials at a cafe. The Pointy Haired Boss(PHB) needs to add/remove items from the menu on a daily basis, so I stored all dishes with MySQL, and created a page which will load all menu items as buttons. When clicked, the button will UPDATE the item, turning it on or off. I need form data to detect which button was pressed, so my query knows which $menuItem to UPDATE. That is the purpose of the hidden fields. <html><head></head> <body> <html><head></head> <body> <?php include("getElement.php"); $keys = array_keys($_POST); echo $keys[0]; echo $keys[1]; //if(isset($_POST["menuItem"])){ //toggleItem($_POST["menuItem"]); //echo print_r(array_keys($_POST));} ?> <form name="b" action="scratchpad.php" method="post" > <input type="hidden" name="b" value="Cajun Gumbo"/> <input type="submit" style="color:blue" value="Cajun Gumbo" /> </form> <form name="a" action="scratchpad.php" method="post" > <input type="hidden" name="a" value="Guacomole Burger"/> <input type="submit" style="color:blue" value="Guacomole Burger" /> </form> </body> </html> Can I get $_POST to identify which button was pressed?

  Read the article

• Unable to render php files in browser

  - by p1

  Hello, I am very new to php, and I am trying to develop a facebook application using php. I am using Joyent as my hosting platform. Currently, I am trying to do some simple scripts in php and then build on them. However I am unable to see any php files being rendered properly in my application. For eg: I have a simple script called phpinfo.php: If I execute this on terminal like php phpinfo.php , I can see all the configurations. However if I try to access the same file as http://xxxxxx.facebook.joyent.us/phpinfo.php, I get : Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request. Even if I rename this file to index.php its still the same. However I am able to access other html files [index.html] on the same location . These are some of my php settings: These are some of the settings: [fbkusoni:~/web/public] aafhe7vh$ php phpinfo.php | grep On allow_url_fopen = On = On auto_globals_jit = On = On enable_dl = On = On file_uploads = On = On ignore_repeated_errors = On = On ignore_repeated_source = On = On implicit_flush = On = On log_errors = On = On register_argc_argv = On = On report_memleaks = On = On y2k_compliance = On = On Multibyte regex (oniguruma) backtrack check = On mysql.allow_persistent = On = On session.bug_compat_warn = On = On session.use_cookies = On = On suhosin.cookie.cryptdocroot = On = On suhosin.cookie.cryptua = On = On suhosin.mt_srand.ignore = On = On suhosin.protectkey = On = On suhosin.server.encode = On = On suhosin.server.strip = On = On suhosin.session.cryptdocroot = On = On suhosin.session.cryptua = On = On suhosin.session.encrypt = On = On suhosin.srand.ignore = On = On suhosin.stealth = On = On The answer might be very naive, but I am just trying to get started and looking for any suggestions regarding this and also using Joyent and cakephp to develop facebook applications. Thanks.

  Read the article

• Output reformatted text within a file included in a JSP

  - by javanix

  I have a few HTML files that I'd like to include via tags in my webapp. Within some of the files, I have pseudo-dynamic code - specially formatted bits of text that, at runtime, I'd like to be resolved to their respective bits of data in a MySQL table. For instance, the HTML file might include a line that says: Welcome, [username]. I want this resolved to (via a logged-in user's data): Welcome, [email protected]. This would be simple to do in a JSP file, but requirements dictate that the files will be created by people who know basic HTML, but not JSP. Simple text-tags like this should be easy enough for me to explain to them, however. I have the code set up to do resolutions like that for strings, but can anyone think of a way to do it across files? I don't actually need to modify the file on disk - just load the content, modify it, and output it w/in the containing JSP file. I've been playing around with trying to load the files into strings via the apache readFileToString, but I can't figure out how to load files from a specific folder within the webapp's content directory without hardcoding it in and having to worry about it breaking if I deploy to a different system in the future.

  Read the article

• How to handle pagination queries properly with mongodb and php?

  - by luckytaxi

  Am I doing this right? I went to look at some old PHP code w/ MySQL and I've managed to get it to work, however I'm wondering if there's a much "cleaner" and "faster" way of accomplishing this. First I would need to get the total number of "documents" $total_documents = $collection->find(array("tags" => $tag, "seeking" => $this->session->userdata('gender'), "gender" => $this->session->userdata('seeking')))->count(); $skip = (int)($docs_per_page * ($page - 1)); $limit = $docs_per_page; $total_pages = ceil($total_documents / $limit); // Query to populate array so I can display with pagination $data['result'] = $collection->find(array("tags" => $tag, "seeking" => $this->session->userdata('gender'), "gender" => $this->session->userdata('seeking')))->limit($limit)->skip($skip)->sort(array("_id" => -1)); My question is, can I run the query in one shot? I'm basically running the same query twice, except the second time I'm passing the value to skip between records.

  Read the article

• tips for fixing bad coding/dev habits ?

  - by dfafa

  i want to become a better coder....so i have decided to sign up for computing science program...maybe a formal education can assist me. i started working on smaller projects to learn but currently i have really bad coding/dev habits which is hindering my productivity as the codebase increases.... i have highlighted them and perhaps someone could make suggestions (or redirect to resources) or a more efficient method. most stuff that i made in the past were web apps. i usually develop with putty + nano...i just love the minimalist feel i use winscp and develop directly on my private web server...too lazy to do it on localhost and upload it later. i dont use subversion control...which one do i need ? sometimes ctrl +z doesn't work well. when i run out of ideas for naming variable, i use swear words instead. i swear a lot when i get stuck....how to deal with anger issue ? my codes look ugly with comments everywhere. would rather use procedural coding finds "thinking" in OO difficult and time consuming i "write first think later". refactors code only if i am getting paid for it. dislikes configuring linux distro, Apache, MySQL, scaling, designing graphics and layouts. does not like writing tests likes working alone. does not like sharing codes. has an econ degree dislikes reading other people's code would rather write it on my own it seems my only true desire is to translate my ideas to a working prototype as fast as possible....it seems like i am very uninterested in the other details...could it be that i am not cut out to be a coder after all ? is going back to study comp sci a bad idea ?

  Read the article

• PDO prepare silently fails

  - by Wabbitseason

  I'm experimenting with PHP's session_set_save_handler and I'd like to use a PDO connection to store session data. I have this function as a callback for write actions: function _write($id, $data) { logger('_WRITE ' . $id . ' ' . $data); try { $access = time(); $sql = 'REPLACE INTO sessions SET id=:id, access=:access, data=:data'; logger('This is the last line in this function that appears in the log.'); $stmt = $GLOBALS['db']->prepare($sql); logger('This never gets logged! :('); $stmt->bindParam(':id', $id, PDO::PARAM_STR); $stmt->bindParam(':access', $access, PDO::PARAM_INT); $stmt->bindParam(':data', $data, PDO::PARAM_STR); $stmt->execute(); $stmt->closeCursor(); return true; } catch (PDOException $e) { logger('This is never executed.'); logger($e->getTraceAsString()); } } The first two log messages always show up, but the third one right after $stmt = $GLOBALS['db']->prepare($sql) never makes it to the log file and there's no trace of an exception either. The sessions db table remains empty. The log message from the _close callback is always present. Here's how I connect to the database: $db = new PDO('mysql:host=' . DBHOST . ';dbname=' . DBNAME, DBUSER, DBPASS); $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); I have PHP 5.2.10. I tried to simply run $GLOBALS['db']->exec($sql) with a "manually prepared" $sql content, but it still failed silently. The query itself is all right I was able to execute it via the db console.

  Read the article

• Temporary storage for keeping data between program iterations?

  - by mr.b

  I am working on an application that works like this: It fetches data from many sources, resulting in pool of about 500,000-1,500,000 records (depends on time/day) Data is parsed Part of data is processed in a way to compare it to pre-existing data (read from database), calculations are made, and stored in database. Resulting dataset that has to be stored in database is, however, much smaller in size (compared to original data set), and ranges from 5,000-50,000 records. This process almost always updates existing data, perhaps adds few more records. Then, data from step 2 should be kept somehow, somewhere, so that next time data is fetched, there is a data set which can be used to perform calculations, without touching pre-existing data in database. I should point out that this data can be lost, it's not irreplaceable (key information can be read from database if needed), but it would speed up the process next time. Application components can (and will be) run off different computers (in the same network), so storage has to be reachable from multiple hosts. I have considered using memcached, but I'm not quite sure should I do so, because one record is usually no smaller than 200 bytes, and if I have 1,500,000 records, I guess that it would amount to over 300 MB of memcached cache... But that doesn't seem scalable to me - what if data was 5x that amount? If it were to consume 1-2 GB of cache only to keep data in between iterations (which could easily happen)? So, the question is: which temporary storage mechanism would be most suitable for this kind of processing? I haven't considered using mysql temporary tables, as I'm not sure if they can persist between sessions, and be used by other hosts in network... Any other suggestion? Something I should consider?

  Read the article

• File Storage for Web Applications: Filesystem vs DB vs NoSQL engines

  - by El Yobo

  I have a web application that stores a lot of user generated files. Currently these are all stored on the server filesystem, which has several downsides for me. When we move "folders" (as defined by our application) we also have to move the files on disk (although this is more due to strange design decisions on the part of the original developers than a requirement of storing things on the filesystem). It's hard to write tests for file system actions; I have a mock filesystem class that logs actions like move, delete etc, without performing them, which more or less does the job, but I don't have 100% confidence in the tests. I will be adding some other jobs which need to access the files from other service to perform additional tasks (e.g. indexing in Solr, generating thumbnails, movie format conversion), so I need to get at the files remotely. Doing this over network shares seems dodgy... Dealing with permissions on the filesystem as sometimes given us problems in the past, although now that we've moved to a pure Linux environment this should be less of an issue. What are the downsides of storing files as BLOBs in MySQL? I guess that it would massively increase the database size and reduce the effectiveness of caches, but are there other problems? Do the same problems exist with NoSQL systems like Cassandra? Does anyone have any other suggestions that might be appropriate?

  Read the article

• Ways to gain a deeper understanding of programming concepts?

  - by MrPlow

  I'm a marketer and have been messing around in PHP/MySQL for years. Recently (the last several months) I've been making my own scripts/programs in Python and I've really enjoyed the whole problem solving process. I've read(skimmed) some books and understand the basics of OOP, polymorphism, etc.. I have a general interest in AI and Natural Language in particular but it seems these things require a masters in Computer Science. My knowledge of math is poor. The last class I took was calculus, and I've forgotten the majority of it. Basically I'm looking for things to learn that will help me think in a more analytic way, and maybe see solutions where I didn't before. Improving my ability to program in Python would be nice too. I don't need to learn a specific language or something for employment, just enjoyment. Although my work often involves web development so some utility would be nice. I don't like learning concepts by just reading them. I need to apply them, even if the examples are contrived. A recommendation of a couple good books or other resources would be nice. :) Apologies if this is too vague/misplaced...

  Read the article

• Rails running multiple delayed_job - lock tables

  - by pepernik

  Hey. I use delayed_job for background processing. I have 8 CPU server, MySQL and I start 7 delayed_job processes RAILS_ENV=production script/delayed_job -n 7 start Q1: I'm wondering is it possible that 2 or more delayed_job processes start processing the same process (the same record-row in the database delayed_jobs). I checked the code of the delayed_job plugin but can not find the lock directive in a way it should be. I think each process should lock the database table before executing an UPDATE on lock_by column. They lock the record simply by updating the locked_by field (UPDATE delayed_jobs SET locked_by...). Is that really enough? No locking needed? Why? I know that UPDATE has higher priority than SELECT but I think this does not have the effect in this case. My understanding of the multy-threaded situation is: Process1: Get waiting job X. [OK] Process2: Get waiting jobs X. [OK] Process1: Update locked_by field. [OK] Process2: Update locked_by field. [OK] Process1: Get waiting job X. [Already processed] Process2: Get waiting jobs X. [Already processed] I think in some cases more jobs can get the same information and can start processing the same process. Q2: Is 7 delayed_jobs a good number for 8CPU server? Why yes/not. Thx 10x!

  Read the article

• Where did Pylons beautiful error handling go? Using Nginx + Paster + Flup#fcgi_thread

  - by Tony

  I need to run my development through nginx due to some complicated subdomain routing rules in my pylons app that wouldn't be handled otherwise. I had been using lighttpd + paster + Flup#scgi_thread and the nice error reporting by Pylons had been working fine in that environment. Yesterday I recompiled Python and MySQL for 64bit, and also switched to Ngix + paster + Flup#fcgi_thread for my development environment. Everything is working great, but I miss the fancy error reports. This is what I get now, and it is a mess compared to what I got used to: http://drp.ly/Iygeg . And here are the pylons/nginx configs. Pylons: [server:main] use = egg:Flup#fcgi_thread host = 0.0.0.0 port = 6500 Nginx: location / { #include /usr/local/nginx/conf/fastcgi.conf; fastcgi_param PATH_INFO $fastcgi_script_name; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param QUERY_STRING $query_string; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; fastcgi_param SERVER_ADDR $server_addr; fastcgi_param SERVER_PORT $server_port; fastcgi_param SERVER_NAME $server_name; fastcgi_param SERVER_PROTOCOL $server_protocol; fastcgi_param REMOTE_ADDR $remote_addr; fastcgi_pass_header Authorization; fastcgi_intercept_errors off; fastcgi_pass 127.0.0.1:6500; }

  Read the article

• Interesting questions related to lighttpd on Amazon EC2

  - by terence410

  This problem appeared today and I have no idea what is going on. Please share you ideas. I have 1 EC2 DB server (MYSQL + NFS File Sharing + Memcached). And I have 3 EC2 Web servers (lighttpd) where it will mounted the NFS folders on the DB server. Everything going smoothly for months but suddenly there is an interesting phenomenon. In every 8 minutes to 10 minutes, PHP file will be unreachable. This will last about 1 minute and then back to normal. Normal files like .html file are unaffected. All servers have the same problem exactly at the same time. I have spent one whole day to analysis the reason. Finally, I find out when the problem appear, the file descriptor of lighttpd suddenly increased a lot. I used ls /proc/1234/fd | wc -l to check the number of fd. The # of fd is around 250 in normal time. However, when the problem appeared, it will be raised to 1500 and then back to normal. It sounds funny, right? Do you have any idea what's going on? ======================== The CPU graph of one of the web server.

  Read the article

< Previous Page | 532 533 534 535 536 537 538 539 540 541 542 543  | Next Page >