Search Results

Search found 19367 results on 775 pages for 'array unique'.

Page 259/775 | < Previous Page | 255 256 257 258 259 260 261 262 263 264 265 266  | Next Page >

  • Drill down rss reader iphone

    - by bing
    Hi everyone, I have made a simple rss reader. The app loads an xml atom file in an array. Now I have added categories to my atom feed, which are first loaded in the array What is the best way to add drill down functionality programmatically. Now only the categories are loaded into the array and displayed. This is the implementation code ..... loading xml file <snip> ..... - (void)parserDidStartDocument:(NSXMLParser *)parser { NSLog(@"found file and started parsing"); } - (void)parser:(NSXMLParser *)parser parseErrorOccurred:(NSError *)parseError { NSString * errorString = [NSString stringWithFormat:@"Unable to download story feed from web site (Error code %i )", [parseError code]]; NSLog(@"error parsing XML: %@", errorString); UIAlertView * errorAlert = [[UIAlertView alloc] initWithTitle:@"Error loading content" message:errorString delegate:self cancelButtonTitle:@"OK" otherButtonTitles:nil]; [errorAlert show]; } - (void)parser:(NSXMLParser *)parser didStartElement:(NSString *)elementName namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qName attributes:(NSDictionary *)attributeDict{ //NSLog(@"found this element: %@", elementName); currentElement = [elementName copy]; if ([elementName isEqualToString:@"entry"]) { // clear out our story item caches... Categoryentry = [[NSMutableDictionary alloc] init]; currentID = [[NSMutableString alloc] init]; currentTitle = [[NSMutableString alloc] init]; currentSummary = [[NSMutableString alloc] init]; currentContent = [[NSMutableString alloc] init]; } } - (void)parser:(NSXMLParser *)parser didEndElement:(NSString *)elementName namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qName{ //NSLog(@"ended element: %@", elementName); if ([elementName isEqualToString:@"entry"]) { // save values to an entry, then store that item into the array... [Categoryentry setObject:currentTitle forKey:@"title"]; [Categoryentry setObject:currentID forKey:@"id"]; [Categoryentry setObject:currentSummary forKey:@"summary"]; [Categoryentry setObject:currentContent forKey:@"content"]; [categories addObject:[Categoryentry copy]]; NSLog(@"adding category: %@", currentTitle); } } - (void)parser:(NSXMLParser *)parser foundCharacters:(NSString *)string{ //NSLog(@"found characters: %@", string); // save the characters for the current item... if ([currentElement isEqualToString:@"title"]) { [currentTitle appendString:string]; } else if ([currentElement isEqualToString:@"id"]) { [currentID appendString:string]; } else if ([currentElement isEqualToString:@"summary"]) { [currentSummary appendString:string]; } else if ([currentElement isEqualToString:@"content"]) { [currentContent appendString:string]; } } - (void)parserDidEndDocument:(NSXMLParser *)parser { [activityIndicator stopAnimating]; [activityIndicator removeFromSuperview]; NSLog(@"all done!"); NSLog(@"categories array has %d entries", [categories count]); [newsTable reloadData]; }

    Read the article

  • how to register the app to open the pdf file in my app in ipad

    - by uttam
    i want to open the pdf file in my app from pdf page, but i am not getting any option of opening the pdf in my app. this my info.plist file <key>CFBundleDevelopmentRegion</key> <string>English</string> <key>CFBundleDocumentTypes</key> <array> <dict> <key>CFBundleTypeName</key> <string>PDF</string> <key>CFBundleTypeRole</key> <string>Viewer</string> <key>CFBundleTypeIconFiles</key> <string>Icon.png</string> <key>LSItemContentTypes</key> <string>com.neosofttech.pdf</string> <key>LSHandlerRank</key> <string>Owner</string> </dict> </array> <key>UTExportedTypeDeclarations</key> <array> <dict> <key>UTTypeConformsTo</key> <array> <string>public.pdf</string> </array> <key>UTTypeDescription</key> <string>PDFReader File</string> <key>UTTypeIdentifier</key> <string>com.neosofttech.pdf</string> <key>UTTypeTagSpecification</key> <dict> <key>public.filename-extension</key> <string>pdf</string> </dict> </dict> pls tell me where i am wrong in this, how can i open the pdf file in my app.

    Read the article

  • Effective Data Validation

    - by John Conde
    What's an effective way to handle data validation, say, from a form submission? Originally I had a bunch of if statements that checked each value and collected invalid values in an array for later retrieval (and listing). // Store errors here $errors = array(); // Hypothetical check if a string is alphanumeric if (!preg_match('/^[a-z\d]+$/i', $fieldvalue)) { $errors[$fieldname] = 'Please only use letters and numbers for your street address'; } // etc... What I did next was create a class that handles various data validation scenarios and store the results in an internal array. After data validation was complete I would check to see if any errors occurred and handle accordingly: class Validation { private $errorList = array(); public function isAlphaNumeric($string, $field, $msg = '') { if (!preg_match('/^[a-z\d]+$/i', $string)) { $this->errorList[$field] = $msg; } } // more methods here public function creditCard($cardNumber, $field, $msg = '') { // Validate credit card number } // more methods here public function hasErrors() { return count($this->errorList); } } /* Client code */ $validate = new Validation(); $validate->isAlphaNumeric($fieldvalue1, $fieldname1, 'Please only use letters and numbers for your street address'); $validate->creditCard($fieldvalue2, $fieldname2, 'Please enter a valid credit card number'); if ($validate->hasErrors()) { // Handle as appropriate } Naturally it didn't take long before this class became bloated with the virtually unlimited types of data to be validated. What I'm doing now is using decorators to separate the different types of data into their own classes and call them only when needed leaving generic validations (i.e. isAlphaNumeric()) in the base class: class Validation { private $errorList = array(); public function isAlphaNumeric($string, $field, $msg = '') { if (!preg_match('/^[a-z\d]+$/i', $string)) { $this->errorList[$field] = $msg; } } // more generic methods here public function setError($field, $msg = '') { $this->errorList[$field] = $msg; } public function hasErrors() { return count($this->errorList); } } class ValidationCreditCard { protected $validate; public function __construct(Validation $validate) { $this->validate = $validate; } public function creditCard($cardNumber, $field, $msg = '') { // Do validation // ... // if there is an error $this->validate->setError($field, $msg); } // more methods here } /* Client code */ $validate = new Validation(); $validate->isAlphaNumeric($fieldvalue, $fieldname, 'Please only use letters and numbers for your street address'); $validateCC = new ValidationCreditCard($validate); $validateCC->creditCard($fieldvalue2, $fieldname2, 'Please enter a valid credit card number'); if ($validate->hasErrors()) { // Handle as appropriate } Am I on the right track? Or did I just complicate data validation more then I needed to?

    Read the article

  • cellForRowAtIndexPath called too late

    - by Mihai Fonoage
    Hi, I am trying to re-load a table every time some data I get from the web is available. This is what I have: SearchDataViewController: - (void)parseDatatXML { parsingDelegate = [[XMLParsingDelegate alloc] init]; parsingDelegate.searchDataController = self; // CONTAINS THE TABLE THAT NEEDS RE-LOADING; ImplementedSearchViewController *searchController = [[ImplementedSearchViewController alloc] initWithNibName:@"ImplementedSearchView" bundle:nil]; ProjectAppDelegate *delegate = [[UIApplication sharedApplication] delegate]; UINavigationController *nav = (UINavigationController *)[delegate.splitViewController.viewControllers objectAtIndex: 0]; NSArray *viewControllers = [[NSArray alloc] initWithObjects:nav, searchController, nil]; self.splitViewController.viewControllers = viewControllers; [viewControllers release]; // PASS A REFERENCE TO THE PARSING DELEGATE SO THAT IT CAN CALL reloadData on the table parsingDelegate.searchViewController = searchController; [searchController release]; // Build the url request used to fetch data ... NSURLRequest *dataURLRequest = [NSURLRequest requestWithURL:[NSURL URLWithString:dataURL]]; parsingDelegate.feedConnection = [[[NSURLConnection alloc] initWithRequest:dataURLRequest delegate:parsingDelegate] autorelease]; } ImplementedSearchViewController: - (NSInteger)tableView:(UITableView *)tableView numberOfRowsInSection:(NSInteger)section { NSLog(@"count = %d", [keys count]); // keys IS A NSMutableArray return [self.keys count]; } - (UITableViewCell *)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath { ... cell.textLabel.text = [keys objectAtIndex:row]; ... } XMLParsingDelgate: -(void) updateSearchTable:(NSArray *)array { ... [self.currentParseBatch addObject:(NSString *)[array objectAtIndex:1]]; // RELOAD TABLE [self.searchViewController.table reloadData]; } - (void)parser:(NSXMLParser *)parser didStartElement:(NSString *)elementName namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qualifiedName attributes:(NSDictionary *)attributeDict { if ([elementName isEqualToString:@"..."]) { self.currentParseBatch = [NSMutableArray array]; searchViewController.keys = self.currentParseBatch; ... } ... } - (void)parser:(NSXMLParser *)parser didEndElement:(NSString *)elementName namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qName { if ([elementName isEqualToString:@"..."]) { ... [self performSelectorOnMainThread:@selector(updateSearchTable:) withObject:array waitUntilDone:NO]; } ... } My problem is that when I debug, the calls go between reloadData and numberOfRowsInSection until the keys array is filled with the last data, time at which the cellForRowAtIndexPath gets called. I wanted the table to be updated for each element I send, one by one, instead of just in the end. Any ideas why this behavior? Thank you!

    Read the article

  • Creating a binary file from an IntelHex in C#

    - by Allek
    I'm trying to create a binary file from a intelHex file. Iside the intelHex file I have data and address to which I should write the data inside the binary file. IntelHex file looks like that :10010000214601360121470136007EFE09D2190140 :100110002146017EB7C20001FF5F16002148011988 :10012000194E79234623965778239EDA3F01B2CAA7 :100130003F0156702B5E712B722B732146013421C7 :00000001FF So I have 4 lines here with data since the last one tells us thats the end of file. Here is what I'm doing to create the file while (!streamReader.EndOfStream) { string temp = String.Empty; int address = 0; line = streamReader.ReadLine(); // Get address for each data address = Convert.ToInt32(line.Substring(3, 4), 16); // Get data from each line temp = line.Substring(7, 2); if (temp == "01") break; else { temp = line.Substring(9, line.Length - 11); string[] array = new string[(temp.Length / 2)]; int j = 0; for (int i = 0; i < array.Length; ++i) { array[i] = temp[j].ToString() + temp[j + 1].ToString(); j = j + 2; } temp = String.Empty; for (int i = 0; i < array.Length; ++i) { temp = temp + Convert.ToChar(Convert.ToInt32(array[i], 16)); } } binaryWriter.Seek(address, SeekOrigin.Begin); binaryWriter.Write(temp); binaryWriter.Flush(); } Console.WriteLine("Done...\nPress any key to exit..."); The problem here is, that data in binary file in some places is not equal to data from the intelHex file. Looks like there is some random data added to the file and I do not know from where. First time I saw that there is an additional data before the data from the intelHex file. For instance first data line starts with 21, but in binary file I have a number 12 before the 21. I do not know what is wrong here. Hope someone can help me or guide me where I can find some usefull informations about creating binary files in C#

    Read the article

  • Zend Framework-where should this root.php file go for MVC?

    - by Joel
    Hi guys, I'm converting over a web-app to use the MVC structure of Zend Framework. I have a root.php include file that contains most of the database info, and some static variables that are used in the program. I'm not sure if some of this should be in the application.ini of in a model that is called by the init() function in a controller, or in the bootstrap or what? Any help would be much appreciated! root.php (include file at the top of every php page): <?php /*** //Configuration file */ ## Site Configuration starts ## define("SITE_ROOT" , dirname(__FILE__)); define("SITE_URL" , "http://localhost/monkeycalendarapp/monkeycalendarapp/public"); define('DB_HOST', "localhost"); define('DB_USER', "root"); define('DB_PASS', "xxx"); define('DB_NAME', "xxxxx"); define("PROJECT_NAME" , "Monkey Mind Manager (beta 2.2)"); //site title define("CALENDAR_WIDTH" , "300"); //left mini calendar width define("CALENDAR_HEIGHT" , "150"); //left mini calendar height $page_title = 'Event List'; $stylesheet_name = 'style.css'; //default stylesheet define("SITE_URL_AJAX" , SITE_URL . "/ajax-tooltip"); define("JQUERY" , SITE_URL . "/jquery-ui-1.7.2"); $a_times = array("12:00","12:30","01:00","01:30","02:00","02:30","03:00","03:30","04:00","04:30","05:00","05:30","06:00","06:30","07:00","07:30","08:00","08:30","09:00","09:30","10:00","10:30","11:00","11:30"); //PTLType Promotional timeline type $a_ptlType= array(1=>"Gigs","To-Do","Completed"); $a_days = array("Su","Mo","Tu","We","Th","Fr","Sa"); $a_timesMerd = array("12:00am","12:30am","01:00am","01:30am","02:00am","02:30am","03:00am","03:30am","04:00am","04:30am","05:00am","05:30am","06:00am","06:30am","07:00am","07:30am","08:00am","08:30am","09:00am","09:30am","10:00am","10:30am","11:00am","11:30am","12:00pm","12:30pm","01:00pm","01:30pm","02:00pm","02:30pm","03:00pm","03:30pm","04:00pm","04:30pm","05:00pm","05:30pm","06:00pm","06:30pm","07:00pm","07:30pm","08:00pm","08:30pm","09:00pm","09:30pm","10:00pm","10:30pm","11:00pm","11:30pm"); //Setting stylesheet for this user. $AMPM=array("am"=>"am","pm"=>"pm"); include(SITE_ROOT . "/includes/functions/general.php"); include(SITE_ROOT . "/includes/db.php"); session_start(); if(isset($_SESSION['userData']['UserID'])) { $s_userID = $_SESSION['userData']['UserID']; } $stylesheet_name = stylesheet(); ini_set('date.timezone', 'GMT'); date_default_timezone_set('GMT'); if($s_userID) { ini_set('date.timezone', $_SESSION['userData']['timezone']); date_default_timezone_set($_SESSION['userData']['timezone']); } ?>

    Read the article

  • What are the rules governing how a bind variable can be used in Postgres and where is this defined?

    - by Craig Miles
    I can have a table and function defined as: CREATE TABLE mytable ( mycol integer ); INSERT INTO mytable VALUES (1); CREATE OR REPLACE FUNCTION myfunction (l_myvar integer) RETURNS mytable AS $$ DECLARE l_myrow mytable; BEGIN SELECT * INTO l_myrow FROM mytable WHERE mycol = l_myvar; RETURN l_myrow; END; $$ LANGUAGE plpgsql; In this case l_myvar acts as a bind variable for the value passed when I call: SELECT * FROM myfunction(1); and returns the row where mycol = 1 If I redefine the function as: CREATE OR REPLACE FUNCTION myfunction (l_myvar integer) RETURNS mytable AS $$ DECLARE l_myrow mytable; BEGIN SELECT * INTO l_myrow FROM mytable WHERE mycol IN (l_myvar); RETURN l_myrow; END; $$ LANGUAGE plpgsql; SELECT * FROM myfunction(1); still returns the row where mycol = 1 However, if I now change the function definition to allow me to pass an integer array and try to this array in the IN clause, I get an error: CREATE OR REPLACE FUNCTION myfunction (l_myvar integer[]) RETURNS mytable AS $$ DECLARE l_myrow mytable; BEGIN SELECT * INTO l_myrow FROM mytable WHERE mycol IN (array_to_string(l_myvar, ',')); RETURN l_myrow; END; $$ LANGUAGE plpgsql; Analysis reveals that although: SELECT array_to_string(ARRAY[1, 2], ','); returns 1,2 as expected SELECT * FROM myfunction(ARRAY[1, 2]); returns the error operator does not exist: integer = text at the line: WHERE mycol IN (array_to_string(l_myvar, ',')); If I execute: SELECT * FROM mytable WHERE mycol IN (1,2); I get the expected result. Given that array_to_string(l_myvar, ',') evaluates to 1,2 as shown, why arent these statements equivalent. From the error message it is something to do with datatypes, but doesnt the IN(variable) construct appear to be behaving differently from the = variable construct? What are the rules here? I know that I could build a statement to EXECUTE, treating everything as a string, to achieve what I want to do, so I am not looking for that as a solution. I do want to understand though what is going on in this example. Is there a modification to this approach to make it work, the particular example being to pass in an array of values to build a dynamic IN clause without resorting to EXECUTE? Thanks in advance Craig

    Read the article

  • Multiple Timers with setTimeInterval

    - by visibleinvisibly
    I am facing a problem with setInterval being used in a loop. I have a function subscribeFeed( ) which takes an array of urls as input. It loops through the url array and subscribes each url to getFeedAutomatically() using a setInterval function. so if three URL's are there in the array, then 3 setInterval's will be called. The problem is 1)how to distinguish which setInterval is called for which URL. 2)it is causing Runtime exception in setInterval( i guess because of closure problem in javascript) //constructor function myfeed(){ this.feedArray = []; } myfeed.prototype.constructor= myfeed; myfeed.prototype.subscribeFeed =function(feedUrl){ var i=0; var url; var count = 0; var _this = this; var feedInfo = { url : [], status : "" }; var urlinfo = []; feedUrl = (feedUrl instanceof Array) ? feedUrl : [feedUrl]; //notifyInterval = (notifyInterval instanceof Array) ? notifyInterval: [notifyInterval]; for (i = 0; i < feedUrl.length; i++) { urlinfo[i] = { url:'', notifyInterval:5000,// Default Notify/Refresh interval for the feed isenable:true, // true allows the feed to be fetched from the URL timerID: null, //default ID is null called : false, position : 0, getFeedAutomatically : function(url){ _this.getFeedUpdate(url); }, }; urlinfo[i].url = feedUrl[i].URL; //overide the default notify interval if(feedUrl[i].NotifyInterval /*&& (feedUrl[i] !=undefined)*/){ urlinfo[i].notifyInterval = feedUrl[i].NotifyInterval; } // Trigger the Feed registered event with the info about URL and status feedInfo.url[i] = feedUrl[i].URL; //Set the interval to get the feed. urlinfo[i].timerID = setInterval(function(){ urlinfo[i].getFeedAutomatically(urlinfo[i].url); }, urlinfo[i].notifyInterval); this.feedArray.push(urlinfo[i]); } } // The getFeedUpate function will make an Ajax request and coninue myfeed.prototype.getFeedUpdate = function( ){ } I am posting the same on jsfiddle http://jsfiddle.net/visibleinvisibly/S37Rj/ Thanking you in advance

    Read the article

  • Defined variables and arrays vs functions in php

    - by Frank Presencia Fandos
    Introduction I have some sort of values that I might want to access several times each page is loaded. I can take two different approaches for accessing them but I'm not sure which one is 'better'. Three already implemented examples are several options for the Language, URI and displaying text that I describe here: Language Right now it is configured in this way: lang() is a function that returns different values depending on the argument. Example: lang("full") returns the current language, "English", while lang() returns the abbreviation of the current language, "en". There are many more options, like lang("select"), lang("selectact"), etc that return different things. The code is too long and irrelevant for the case so if anyone wants it just ask for it. Url The $Url array also returns different values depending on the request. The whole array is fully defined in the beginning of the page and used to get shorter but accurate links of the current page. Example: $Url['full'] would return "http://mypage.org/path/to/file.php?page=1" and $Url['file'] would return "file.php". It's useful for action="" within the forms and many other things. There are more values for $Url['folder'], $Url['file'], etc. Same thing about the code, if wanted, just request it. Text [You can skip this section] There's another array called $Text that is defined in the same way than $Url. The whole array is defined at the beginning, making a mysql call and defining all $Text[$i] for current page with a while loop. I'm not sure if this is more efficient than multiple calls for a single mysql cell. Example: $Text['54'] returns "This is just a test array!" which this could perfectly be implemented with a function like text(54). Question With the 3 examples you can see that I use different methods to do almost the same function (no pun intended), but I'm not sure which one should become the standard one for my code. I could create a function called url() and other called text() to output what I want. I think that working with functions in those cases is better, but I'm not sure why. So I'd really appreciate your opinions and advice. Should I mix arrays and functions in the way I described or should I just use funcions? Please, base your answer in this: The source needs to be readable and reusable by other developers Resource consumption (processing, time and memory). The shorter the code the better. The more you explain the reasons the better. Thank you PS, now I know the differences between $Url and $Uri.

    Read the article

  • c++ recursion with arrays

    - by Sam
    I have this project that I'm working on for a class, and I'm not looking for the answer, but maybe just a few tips since I don't feel like I'm using true recursion. The problem involves solving the game of Hi Q or "peg solitaire" where you want the end result to be one peg remaining (it's played with a triangular board and one space is open at the start.) I've represented the board with a simple array, each index being a spot and having a value of 1 if it has a peg, and 0 if it doesn't and also the set of valid moves with a 2 dimensional array that is 36, 3 (each move set contains 3 numbers; the peg you're moving, the peg it hops over, and the destination index.) So my problem is that in my recursive function, I'm using a lot of iteration to determine things like which space is open (or has a value of 0) and which move to use based on which space is open by looping through the arrays. Secondly, I don't understand how you would then backtrack with recursion, in the event that an incorrect move was made and you wanted to take that move back in order to choose a different one. This is what I have so far; the "a" array represents the board, the "b" array represents the moves, and the "c" array was my idea of a reminder as to which moves I used already. The functions used are helper functions that basically just loop through the arrays to find an empty space and corresponding move. : void solveGame(int a[15], int b[36][3], int c[15][3]){ int empSpace; int moveIndex; int count = 0; if(pegCount(a) < 2){ return; } else{ empSpace = findEmpty(a); moveIndex = chooseMove( a, b, empSpace); a[b[moveIndex][0]] = 0; a[b[moveIndex][1]] = 0; a[b[moveIndex][2]] = 1; c[count][0] = b[moveIndex][0]; c[count][1] = b[moveIndex][1]; c[count][2] = b[moveIndex][2]; solveGame(a,b,c); } }

    Read the article

  • Finding open contiguous blocks of time for every day of a month, fast

    - by Chris
    I am working on a booking availability system for a group of several venues, and am having a hard time generating the availability of time blocks for days in a given month. This is happening server-side in PHP, but the concept itself is language agnostic -- I could be doing this in JS or anything else. Given a venue_id, month, and year (6/2012 for example), I have a list of all events occurring in that range at that venue, represented as unix timestamps start and end. This data comes from the database. I need to establish what, if any, contiguous block of time of a minimum length (different per venue) exist on each day. For example, on 6/1 I have an event between 2:00pm and 7:00pm. The minimum time is 5 hours, so there's a block open there from 9am - 2pm and another between 7pm and 12pm. This would continue for the 2nd, 3rd, etc... every day of June. Some (most) of the days have nothing happening at all, some have 1 - 3 events. The solution I came up with works, but it also takes waaaay too long to generate the data. Basically, I loop every day of the month and create an array of timestamps for each 15 minutes of that day. Then, I loop the time spans of events from that day by 15 minutes, marking any "taken" timeslot as false. Remaining, I have an array that contains timestamp of free time vs. taken time: //one day's array after processing through loops (not real timestamps) array( 12345678=>12345678, // <--- avail 12345878=>12345878, 12346078=>12346078, 12346278=>false, // <--- not avail 12346478=>false, 12346678=>false, 12346878=>false, 12347078=>12347078, // <--- avail 12347278=>12347278 ) Now I would need to loop THIS array to find continuous time blocks, then check to see if they are long enough (each venue has a minimum), and if so then establish the descriptive text for their start and end (i.e. 9am - 2pm). WHEW! By the time all this looping is done, the user has grown bored and wandered off to Youtube to watch videos of puppies; it takes ages to so examine 30 or so days. Is there a faster way to solve this issue? To summarize the problem, given time ranges t1 and t2 on day d, how can I determine the remaining time left in d that is longer than the minimum time block m. This data is assembled on demand via AJAX as the user moves between calendar months. Results are cached per-page-load, so if the user goes to July a second time, the data that was generated the first time would be reused. Any other details that would help, let me know. Edit Per request, the database structure (or the part that is relevant here) *events* id (bigint) title (varchar) *event_times* id (bigint) event_id (bigint) venue_id (bigint) start (bigint) end (bigint) *venues* id (bigint) name (varchar) min_block (int) min_start (varchar) max_start (varchar)

    Read the article

  • Yii file upload issue

    - by user1289853
    Couple of months ago,I developed a simple app using YII,one of the feature was to upload file. The feature was working well in my dev machine,couple of days ago client found the file upload feature is not working in his server since deployment. And after that I test my dev machine that was not working too. My controller looks: public function actionEntry() { if (!Yii::app()->user->isGuest) { $model = new TrackForm; if (isset($_POST['TrackForm'])) { $entry = new Track; try { $entry->product_image = $_POST['TrackForm']['product_image']; $entry->product_image = CUploadedFile::getInstance($model, 'product_image'); if ($entry->save()) { if ($entry->product_image) { $entry->product_image->saveAs($entry->product_image->name, '/trackshirt/uploads'); } } $this->render('success', array('model' => $model)); // redirect to success page } } catch (Exception $e) { echo 'Caught exception: ', $e->getMessage(), "\n"; } } else { $this->render('entry', array('model' => $model)); } } } Model is like below: <?php class Track extends CActiveRecord { public static function model($className=__CLASS__) { return parent::model($className); } public function tableName() { return 'product_details'; } } My view looks: <?php $form = $this->beginWidget('CActiveForm', array( 'id' => 'hide-form', 'enableClientValidation' => true, 'clientOptions' => array( 'validateOnSubmit' => true, ), 'htmlOptions' => array('enctype' => 'multipart/form-data'), )); ?> <p class="auto-style2"><strong>Administration - Add New Product</strong></p> <table align="center" style="width: 650px"><td class="auto-style3" style="width: 250px">Product Image</td> <td> <?php echo $form->activeFileField($model, 'product_image'); ?> </td> </tr> </table> <p class="auto-style1"> <div style="margin-leftL:-100px;"> <?php echo CHtml::submitButton('Submit New Product Form'); ?> </div> <?php $this->endWidget(); ?> Any idea where is the problem?I tried to debug it but every time it returns Null. Thanks.

    Read the article

  • Lithium: Run filter after find() to format output

    - by Housni
    I wanted to specify the output of a field from within my model so I added a date key to my $_schema: models/Tags.php <?php protected $_schema = array( 'id' => array('type' => 'integer', 'key' => 'primary'), 'title' => array('type' => 'string'), 'created' => array('type' => 'integer', 'date' => 'F jS, Y - g:i a'), 'modified' => array('type' => 'integer') ); ?> I store my time as an unsigned integer in the db (output of time()). I want my base model to format any field that has the date key for output. I thought the best place to do that would be right after a find: extensions/data/Model.php <?php static::applyFilter('find', function($self, $params, $chain) { $schema = $self::schema(); $entity = $chain->next($self, $params, $chain); foreach ($schema as $field => $options) { if (array_key_exists('date', $options)) { //format as a date $params['data'][$field] = $entity->formatDate($field, $options['date']); } } return $entity; }); public function formatdate($entity, $field, $format, $timezone = 'Asia/Colombo') { $dt = new \DateTime(); $tz = new \DateTimeZone($timezone); $dt->setTimestamp($entity->$field); $dt->setTimezone($tz); return $dt->format($format); } ?> This doesn't seem to be working. When I execute a find all, this filter seems to get hit twice. The first time, $entity contains a count() of the results and only on the second hit does it contain the Records object. What am I doing wrong? How do I alter this so that simply doing <?= $tag->created; ?> in my view will format the date the way I want? This, essentially, needs to be an 'after filter', of sorts. EDIT If I can find a way to access the current model entity object (not the full namespaced path, $self contains that), I can probably solve my problem.

    Read the article

  • Drupal menu permissions question

    - by Luke
    I'm creating an admin module for my client that gives then access to some administration functionality concerning their content. I'm starting off my adding some permissions in my module by implementing hook_perm: function mymodule_perm() { return array( 'manage projects', ); } I can then create my menu by adding to the admin section that already exists: function mymodule_menu() { $items['admin/projects'] = array( 'title' => 'Projects', 'description' => 'Manage your projects.', 'page callback' => 'manage_projects_overview', 'access callback' => 'user_access', 'access arguments' => array('manage projects'), 'type' => MENU_NORMAL_ITEM, 'weight' => -100, ); $items['admin/projects/add'] = array( 'title' => 'Add project', 'access arguments' => array('manage projects'), 'page callback' => 'mymodule_projects_add', 'type' => MENU_NORMAL_ITEM, 'weight' => 1, ); return $items; } This will add a Projects section to the Administration area with an Add project sub section. All good. The behavior I want is that my client can only see the Projects section when they log in. I've accomplished this by ticking the "manage projects" permission for authenticated users. Now to give my client actual access to the Administration area I also need to tick "access administration pages" under the "system module" in the users permissions section. This works great, when I log in as my client I can only see the Projects section in the Administration area. There is one thing though, I my Navigation menu shown in the left column I can see the following items: - Administer - Projects - Content management - Site building - Site configuration - User management I was expecting only the see Administer and Projects, not the other ones. When I click e.g. Content Management I get a Content Management titled page with no options. Same for Site Building, Site Configuration and User Management. What's really odd is that Reports is not being shown which is also a top level Administration section. Why are these other items, besides my Projects section, being shown and how can I make them stop from appearing if I'm not logged in as administrator?

    Read the article

  • Inventory count in CakePHP

    - by metrobalderas
    We are developing an inventory tracking system. Basically we've got an order table in which orders are placed. When an order is payed, the status changes from 0 to 1. This table has multiple children in another table order_items. This is the main structure. CREATE TABLE order( id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY, user_id INT UNSIGNED, status INT(1), total INT UNSIGNED ); CREATE TABLE order_items( id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY, order_id INT UNSIGNED, article_id INT UNSIGNED, size enum('s', 'm', 'l', 'xl'), quantity INT UNSIGNED ); Now, we've got a stocks table with similar architecture for the acquisitions. This is the structure. CREATE TABLE stock( id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY, article_id INT UNSIGNED ); CREATE TABLE stock_items( id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY, stock_id INT UNSIGNED, size enum('s', 'm', 'l', 'xl'), quantity INT(2) ); The main difference is that stocks has no status field. What we are looking for is a way to sum each article size from stock_items, then sum each article size from order_items where Order.status = 1 and substract both these items to find our current inventory. This is the table we want to get from a single query: Size | Stocks | Sales | Available s | 10 | 3 | 7 m | 15 | 13 | 2 l | 7 | 4 | 3 Initially we thought abouth using complex find conditions, but perhaps that's the wrong approach. Also, since it's not a direct join, it turns out to be quite hard. This is the code we have to retrieve the stock's total for each item. function stocks_total($id){ $find = $this->StockItem->find('all', array( 'conditions' => array( 'StockItem.stock_id' => $this->find('list', array('conditions' => array('Stock.article_id' => $id))) ), 'fields' => array_merge( array( 'SUM(StockItem.cantidad) as total' ), array_keys($this->StockItem->_schema) ), 'group' => 'StockItem.size', 'order' => 'FIELD(StockItem.size, \'s\', \'m\' ,\'l\' ,\'xl\') ASC' )); return $find; } Thanks.

    Read the article

  • Line Graph CGPoints from NSMutableArray

    - by Mattog1456
    I have been trying to adapt the code from the accelerometer example such that when the user depresses a uibutton a point is added to a line graph. Working on the converting two floats, which are the result of calculate as below into a CGPoint and converting the CGPoint into an NSValue and then adding this to a NSMutableArray with the following -(IBAction)calculate:(id)sender { self.points = [NSMutableArray array]; CGPoint pt = CGPointMake(d, c); [self.points addObject:[NSValue valueWithCGPoint:pt]]; NSLog(@"%@", NSStringFromCGPoint(pt)); NSLog(@"%@", [NSString stringWithFormat:@"%d points", self.points.count ]); } But for some reason I am only getting one object stored in the array, it seems everytime push the calculate button the object pt gets overwritten, on the plus side it has the correct x,y coords. Any ideas on this one? UPDATE Removed self.points = [NSMutableArray array]; and placed it in view did load, also set the first points to 0,0. so that is working ok. Now the next problem is that I have a Graph subclass where the CG Drawing is taking place. I am trying to figure out a simple way to be able to access the above NSMutableArray which is in a ViewController class from the graph class. Am so close to the end but am really stuck, any help would be great. Still trying to draw a line graph on a UIView which is on a UIScrollview. The draw rect method is in the UIView Subclass and everything is working there, I have gridlines and labels on the axis and I can draw manually onto it. But the problem I have is that I cannot read the NSMutableArray of the CGPoints, which are the x and y coords. The ViewController performs a calculation and the results are written to the NSMutable array and this is all working fine as well, I can see the CGpoints and their values being written with NSLogs in the ViewController. I have tried various ways to set the NSMutableArray up as a global but to no avail, everything runs but while I can see the points being written in the ViewController they are just not visible to the UIView Subclass. I have also tried to use the addObserver and observeValueForKeyPath methods and once again while everything runs the subclass cannot see the array. Any ideas, suggestions, tips or thoughts would be great

    Read the article

  • XNA 4.0 draw a cube with DrawUserIndexedPrimitives method [on hold]

    - by Leggy7
    EDIT Since I read what Mark H suggested (thanks a lot, I found it very useful) I think my question can become clearer structured this way: Using XNA 4.0, I'm trying to draw a cube. Im using this method: GraphicsDevice.DrawUserIndexedPrimitives<VertexPositionColor>( PrimitiveType.LineList, primitiveList, 0, // vertex buffer offset to add to each element of the index buffer 8, // number of vertices in pointList lineListIndices, // the index buffer 0, // first index element to read 7 // number of primitives to draw ); I got the code sample from this page which simply draw a serie of triangles. I want to modify this code in order to draw a cube. I was able to slitghly move the camera so I can have the perception of solidity, I set the vertex array to contain the 8 points defining a cube. But I can't fully understand how many primitives I have to draw (last parameter) for each of PrimitiveType. So, I wasn't able to draw the cube (just some of the edges in a non-defined order). More in detail: to build the vertex index list, the sample used // Initialize an array of indices of type short. lineListIndices = new short[(points * 2) - 2]; // Populate the array with references to indices in the vertex buffer for (int i = 0; i < points - 1; i++) { lineListIndices[i * 2] = (short)(i); lineListIndices[(i * 2) + 1] = (short)(i + 1); } I'm ashamed to say I cannot do the same in the case of a cube. what has to be the size of the lineListIndices? how should I populate it? In which order? And how do these things change when I use a different PrimitiveType? In the code sample there are also another couple of calls which I cannot fully understand, which are: // Initialize the vertex buffer, allocating memory for each vertex. vertexBuffer = new VertexBuffer(graphics.GraphicsDevice, vertexDeclaration, points, BufferUsage.None); // Set the vertex buffer data to the array of vertices. vertexBuffer.SetData<VertexPositionColor>(pointList); and vertexDeclaration = new VertexDeclaration(new VertexElement[] { new VertexElement(0, VertexElementFormat.Vector3, VertexElementUsage.Position, 0), new VertexElement(12, VertexElementFormat.Color, VertexElementUsage.Color, 0) } ); that is, for VertexBuffer and VertexDeclaration I could not find and significant (monkey-like) guide. I reported them too because I think they could be involded in understanding things. I think I also have to understand something related to the order the vertexes are stored in the array. But actually I have no clue of what I should learn to have this function drawing a cube. So, if anybody could point me to the right direction, it wil be appreciated. Hope to have made myself clear this time

    Read the article

  • mysql_query() returns returns true, but mysql_num_rows() and mysql_fetch_array() give "not a valid r

    - by zlance4012
    Here is the code in question: -----From index.php----- require_once('includes/DbConnector.php'); // Create an object (instance) of the DbConnector $connector = new DbConnector(); // Execute the query to retrieve articles $query1 = "SELECT id, title FROM articles ORDER BY id DESC LIMIT 0,5"; $result = $connector-query($query1); echo "vardump1:"; var_dump($result); echo "\n"; /(!line 17!)/ echo "Number of rows in the result of the query:".mysql_num_rows($result)."\n"; // Get an array containing the results. // Loop for each item in that array while ($row = $connector-fetchArray($result)){ echo ' '; echo $row['title']; echo ' '; -----end index.php----- -----included DbConnector.php----- $settings = SystemComponent::getSettings(); // Get the main settings from the array we just loaded $host = $settings['dbhost']; $db = $settings['dbname']; $user = $settings['dbusername']; $pass = $settings['dbpassword']; // Connect to the database $this-link = mysql_connect($host, $user, $pass); mysql_select_db($db); register_shutdown_function(array(&$this, 'close')); } //end constructor //* Function: query, Purpose: Execute a database query * function query($query) { echo "Query Statement: ".$query."\n"; $this-theQuery = $query; return mysql_query($query, $this-link) or die(mysql_error()); } //* Function: fetchArray, Purpose: Get array of query results * function fetchArray($result) { echo "<|"; var_dump($result); echo "| \n"; /(!line 50!)/$res= mysql_fetch_array($result) or die(mysql_error()); echo $res['id']."-".$res['title']."-".$res['imagelink']."-".$res['text']; return $res; } -----end DbConnector.php----- -----Output----- Query Statement: SELECT id, title FROM articles ORDER BY id DESC LIMIT 0,5 vardump1:bool(true) PHP Error Message Warning: mysql_num_rows(): supplied argument is not a valid MySQL result resource in /path to/index.php on line 17 Number of rows in the result of the query: <|bool(true) | PHP Error Message Warning: mysql_fetch_array(): supplied argument is not a valid MySQL result resource in /path to/DbConnector.php on line 50

    Read the article

  • WCF WS-Security and WSE Nonce Authentication

    - by Rick Strahl
    WCF makes it fairly easy to access WS-* Web Services, except when you run into a service format that it doesn't support. Even then WCF provides a huge amount of flexibility to make the service clients work, however finding the proper interfaces to make that happen is not easy to discover and for the most part undocumented unless you're lucky enough to run into a blog, forum or StackOverflow post on the matter. This is definitely true for the Password Nonce as part of the WS-Security/WSE protocol, which is not natively supported in WCF. Specifically I had a need to create a WCF message on the client that includes a WS-Security header that looks like this from their spec document:<soapenv:Header> <wsse:Security soapenv:mustUnderstand="1" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"> <wsse:UsernameToken wsu:Id="UsernameToken-8" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"> <wsse:Username>TeStUsErNaMe1</wsse:Username> <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText" >TeStPaSsWoRd1</wsse:Password> <wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" >f8nUe3YupTU5ISdCy3X9Gg==</wsse:Nonce> <wsu:Created>2011-05-04T19:01:40.981Z</wsu:Created> </wsse:UsernameToken> </wsse:Security> </soapenv:Header> Specifically, the Nonce and Created keys are what WCF doesn't create or have a built in formatting for. Why is there a nonce? My first thought here was WTF? The username and password are there in clear text, what does the Nonce accomplish? The Nonce and created keys are are part of WSE Security specification and are meant to allow the server to detect and prevent replay attacks. The hashed nonce should be unique per request which the server can store and check for before running another request thus ensuring that a request is not replayed with exactly the same values. Basic ServiceUtl Import - not much Luck The first thing I did when I imported this service with a service reference was to simply import it as a Service Reference. The Add Service Reference import automatically detects that WS-Security is required and appropariately adds the WS-Security to the basicHttpBinding in the config file:<?xml version="1.0" encoding="utf-8" ?> <configuration> <system.serviceModel> <bindings> <basicHttpBinding> <binding name="RealTimeOnlineSoapBinding"> <security mode="Transport" /> </binding> <binding name="RealTimeOnlineSoapBinding1" /> </basicHttpBinding> </bindings> <client> <endpoint address="https://notarealurl.com:443/services/RealTimeOnline" binding="basicHttpBinding" bindingConfiguration="RealTimeOnlineSoapBinding" contract="RealTimeOnline.RealTimeOnline" name="RealTimeOnline" /> </client> </system.serviceModel> </configuration> If if I run this as is using code like this:var client = new RealTimeOnlineClient(); client.ClientCredentials.UserName.UserName = "TheUsername"; client.ClientCredentials.UserName.Password = "ThePassword"; … I get nothing in terms of WS-Security headers. The request is sent, but the the binding expects transport level security to be applied, rather than message level security. To fix this so that a WS-Security message header is sent the security mode can be changed to: <security mode="TransportWithMessageCredential" /> Now if I re-run I at least get a WS-Security header which looks like this:<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"> <s:Header> <o:Security s:mustUnderstand="1" xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"> <u:Timestamp u:Id="_0"> <u:Created>2012-11-24T02:55:18.011Z</u:Created> <u:Expires>2012-11-24T03:00:18.011Z</u:Expires> </u:Timestamp> <o:UsernameToken u:Id="uuid-18c215d4-1106-40a5-8dd1-c81fdddf19d3-1"> <o:Username>TheUserName</o:Username> <o:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText" >ThePassword</o:Password> </o:UsernameToken> </o:Security> </s:Header> Closer! Now the WS-Security header is there along with a timestamp field (which might not be accepted by some WS-Security expecting services), but there's no Nonce or created timestamp as required by my original service. Using a CustomBinding instead My next try was to go with a CustomBinding instead of basicHttpBinding as it allows a bit more control over the protocol and transport configurations for the binding. Specifically I can explicitly specify the message protocol(s) used. Using configuration file settings here's what the config file looks like:<?xml version="1.0"?> <configuration> <system.serviceModel> <bindings> <customBinding> <binding name="CustomSoapBinding"> <security includeTimestamp="false" authenticationMode="UserNameOverTransport" defaultAlgorithmSuite="Basic256" requireDerivedKeys="false" messageSecurityVersion="WSSecurity10WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10"> </security> <textMessageEncoding messageVersion="Soap11"></textMessageEncoding> <httpsTransport maxReceivedMessageSize="2000000000"/> </binding> </customBinding> </bindings> <client> <endpoint address="https://notrealurl.com:443/services/RealTimeOnline" binding="customBinding" bindingConfiguration="CustomSoapBinding" contract="RealTimeOnline.RealTimeOnline" name="RealTimeOnline" /> </client> </system.serviceModel> <startup> <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/> </startup> </configuration> This ends up creating a cleaner header that's missing the timestamp field which can cause some services problems. The WS-Security header output generated with the above looks like this:<s:Header> <o:Security s:mustUnderstand="1" xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"> <o:UsernameToken u:Id="uuid-291622ca-4c11-460f-9886-ac1c78813b24-1"> <o:Username>TheUsername</o:Username> <o:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText" >ThePassword</o:Password> </o:UsernameToken> </o:Security> </s:Header> This is closer as it includes only the username and password. The key here is the protocol for WS-Security:messageSecurityVersion="WSSecurity10WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10" which explicitly specifies the protocol version. There are several variants of this specification but none of them seem to support the nonce unfortunately. This protocol does allow for optional omission of the Nonce and created timestamp provided (which effectively makes those keys optional). With some services I tried that requested a Nonce just using this protocol actually worked where the default basicHttpBinding failed to connect, so this is a possible solution for access to some services. Unfortunately for my target service that was not an option. The nonce has to be there. Creating Custom ClientCredentials As it turns out WCF doesn't have support for the Digest Nonce as part of WS-Security, and so as far as I can tell there's no way to do it just with configuration settings. I did a bunch of research on this trying to find workarounds for this, and I did find a couple of entries on StackOverflow as well as on the MSDN forums. However, none of these are particularily clear and I ended up using bits and pieces of several of them to arrive at a working solution in the end. http://stackoverflow.com/questions/896901/wcf-adding-nonce-to-usernametoken http://social.msdn.microsoft.com/Forums/en-US/wcf/thread/4df3354f-0627-42d9-b5fb-6e880b60f8ee The latter forum message is the more useful of the two (the last message on the thread in particular) and it has most of the information required to make this work. But it took some experimentation for me to get this right so I'll recount the process here maybe a bit more comprehensively. In order for this to work a number of classes have to be overridden: ClientCredentials ClientCredentialsSecurityTokenManager WSSecurityTokenizer The idea is that we need to create a custom ClientCredential class to hold the custom properties so they can be set from the UI or via configuration settings. The TokenManager and Tokenizer are mainly required to allow the custom credentials class to flow through the WCF pipeline and eventually provide custom serialization. Here are the three classes required and their full implementations:public class CustomCredentials : ClientCredentials { public CustomCredentials() { } protected CustomCredentials(CustomCredentials cc) : base(cc) { } public override System.IdentityModel.Selectors.SecurityTokenManager CreateSecurityTokenManager() { return new CustomSecurityTokenManager(this); } protected override ClientCredentials CloneCore() { return new CustomCredentials(this); } } public class CustomSecurityTokenManager : ClientCredentialsSecurityTokenManager { public CustomSecurityTokenManager(CustomCredentials cred) : base(cred) { } public override System.IdentityModel.Selectors.SecurityTokenSerializer CreateSecurityTokenSerializer(System.IdentityModel.Selectors.SecurityTokenVersion version) { return new CustomTokenSerializer(System.ServiceModel.Security.SecurityVersion.WSSecurity11); } } public class CustomTokenSerializer : WSSecurityTokenSerializer { public CustomTokenSerializer(SecurityVersion sv) : base(sv) { } protected override void WriteTokenCore(System.Xml.XmlWriter writer, System.IdentityModel.Tokens.SecurityToken token) { UserNameSecurityToken userToken = token as UserNameSecurityToken; string tokennamespace = "o"; DateTime created = DateTime.Now; string createdStr = created.ToString("yyyy-MM-ddThh:mm:ss.fffZ"); // unique Nonce value - encode with SHA-1 for 'randomness' // in theory the nonce could just be the GUID by itself string phrase = Guid.NewGuid().ToString(); var nonce = GetSHA1String(phrase); // in this case password is plain text // for digest mode password needs to be encoded as: // PasswordAsDigest = Base64(SHA-1(Nonce + Created + Password)) // and profile needs to change to //string password = GetSHA1String(nonce + createdStr + userToken.Password); string password = userToken.Password; writer.WriteRaw(string.Format( "<{0}:UsernameToken u:Id=\"" + token.Id + "\" xmlns:u=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd\">" + "<{0}:Username>" + userToken.UserName + "</{0}:Username>" + "<{0}:Password Type=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText\">" + password + "</{0}:Password>" + "<{0}:Nonce EncodingType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary\">" + nonce + "</{0}:Nonce>" + "<u:Created>" + createdStr + "</u:Created></{0}:UsernameToken>", tokennamespace)); } protected string GetSHA1String(string phrase) { SHA1CryptoServiceProvider sha1Hasher = new SHA1CryptoServiceProvider(); byte[] hashedDataBytes = sha1Hasher.ComputeHash(Encoding.UTF8.GetBytes(phrase)); return Convert.ToBase64String(hashedDataBytes); } } Realistically only the CustomTokenSerializer has any significant code in. The code there deals with actually serializing the custom credentials using low level XML semantics by writing output into an XML writer. I can't take credit for this code - most of the code comes from the MSDN forum post mentioned earlier - I made a few adjustments to simplify the nonce generation and also added some notes to allow for PasswordDigest generation. Per spec the nonce is nothing more than a unique value that's supposed to be 'random'. I'm thinking that this value can be any string that's unique and a GUID on its own probably would have sufficed. Comments on other posts that GUIDs can be potentially guessed are highly exaggerated to say the least IMHO. To satisfy even that aspect though I added the SHA1 encryption and binary decoding to give a more random value that would be impossible to 'guess'. The original example from the forum post used another level of encoding and decoding to string in between - but that really didn't accomplish anything but extra overhead. The header output generated from this looks like this:<s:Header> <o:Security s:mustUnderstand="1" xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"> <o:UsernameToken u:Id="uuid-f43d8b0d-0ebb-482e-998d-f544401a3c91-1" xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"> <o:Username>TheUsername</o:Username> <o:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">ThePassword</o:Password> <o:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" >PjVE24TC6HtdAnsf3U9c5WMsECY=</o:Nonce> <u:Created>2012-11-23T07:10:04.670Z</u:Created> </o:UsernameToken> </o:Security> </s:Header> which is exactly as it should be. Password Digest? In my case the password is passed in plain text over an SSL connection, so there's no digest required so I was done with the code above. Since I don't have a service handy that requires a password digest,  I had no way of testing the code for the digest implementation, but here is how this is likely to work. If you need to pass a digest encoded password things are a little bit trickier. The password type namespace needs to change to: http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#Digest and then the password value needs to be encoded. The format for password digest encoding is this: Base64(SHA-1(Nonce + Created + Password)) and it can be handled in the code above with this code (that's commented in the snippet above): string password = GetSHA1String(nonce + createdStr + userToken.Password); The entire WriteTokenCore method for digest code looks like this:protected override void WriteTokenCore(System.Xml.XmlWriter writer, System.IdentityModel.Tokens.SecurityToken token) { UserNameSecurityToken userToken = token as UserNameSecurityToken; string tokennamespace = "o"; DateTime created = DateTime.Now; string createdStr = created.ToString("yyyy-MM-ddThh:mm:ss.fffZ"); // unique Nonce value - encode with SHA-1 for 'randomness' // in theory the nonce could just be the GUID by itself string phrase = Guid.NewGuid().ToString(); var nonce = GetSHA1String(phrase); string password = GetSHA1String(nonce + createdStr + userToken.Password); writer.WriteRaw(string.Format( "<{0}:UsernameToken u:Id=\"" + token.Id + "\" xmlns:u=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd\">" + "<{0}:Username>" + userToken.UserName + "</{0}:Username>" + "<{0}:Password Type=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#Digest\">" + password + "</{0}:Password>" + "<{0}:Nonce EncodingType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary\">" + nonce + "</{0}:Nonce>" + "<u:Created>" + createdStr + "</u:Created></{0}:UsernameToken>", tokennamespace)); } I had no service to connect to to try out Digest auth - if you end up needing it and get it to work please drop a comment… How to use the custom Credentials The easiest way to use the custom credentials is to create the client in code. Here's a factory method I use to create an instance of my service client:  public static RealTimeOnlineClient CreateRealTimeOnlineProxy(string url, string username, string password) { if (string.IsNullOrEmpty(url)) url = "https://notrealurl.com:443/cows/services/RealTimeOnline"; CustomBinding binding = new CustomBinding(); var security = TransportSecurityBindingElement.CreateUserNameOverTransportBindingElement(); security.IncludeTimestamp = false; security.DefaultAlgorithmSuite = SecurityAlgorithmSuite.Basic256; security.MessageSecurityVersion = MessageSecurityVersion.WSSecurity10WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10; var encoding = new TextMessageEncodingBindingElement(); encoding.MessageVersion = MessageVersion.Soap11; var transport = new HttpsTransportBindingElement(); transport.MaxReceivedMessageSize = 20000000; // 20 megs binding.Elements.Add(security); binding.Elements.Add(encoding); binding.Elements.Add(transport); RealTimeOnlineClient client = new RealTimeOnlineClient(binding, new EndpointAddress(url)); // to use full client credential with Nonce uncomment this code: // it looks like this might not be required - the service seems to work without it client.ChannelFactory.Endpoint.Behaviors.Remove<System.ServiceModel.Description.ClientCredentials>(); client.ChannelFactory.Endpoint.Behaviors.Add(new CustomCredentials()); client.ClientCredentials.UserName.UserName = username; client.ClientCredentials.UserName.Password = password; return client; } This returns a service client that's ready to call other service methods. The key item in this code is the ChannelFactory endpoint behavior modification that that first removes the original ClientCredentials and then adds the new one. The ClientCredentials property on the client is read only and this is the way it has to be added.   Summary It's a bummer that WCF doesn't suport WSE Security authentication with nonce values out of the box. From reading the comments in posts/articles while I was trying to find a solution, I found that this feature was omitted by design as this protocol is considered unsecure. While I agree that plain text passwords are rarely a good idea even if they go over secured SSL connection as WSE Security does, there are unfortunately quite a few services (mosly Java services I suspect) that use this protocol. I've run into this twice now and trying to find a solution online I can see that this is not an isolated problem - many others seem to have struggled with this. It seems there are about a dozen questions about this on StackOverflow all with varying incomplete answers. Hopefully this post provides a little more coherent content in one place. Again I marvel at WCF and its breadth of support for protocol features it has in a single tool. And even when it can't handle something there are ways to get it working via extensibility. But at the same time I marvel at how freaking difficult it is to arrive at these solutions. I mean there's no way I could have ever figured this out on my own. It takes somebody working on the WCF team or at least being very, very intricately involved in the innards of WCF to figure out the interconnection of the various objects to do this from scratch. Luckily this is an older problem that has been discussed extensively online and I was able to cobble together a solution from the online content. I'm glad it worked out that way, but it feels dirty and incomplete in that there's a whole learning path that was omitted to get here… Man am I glad I'm not dealing with SOAP services much anymore. REST service security - even when using some sort of federation is a piece of cake by comparison :-) I'm sure once standards bodies gets involved we'll be right back in security standard hell…© Rick Strahl, West Wind Technologies, 2005-2012Posted in WCF  Web Services   Tweet !function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs"); (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/plusone.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();

    Read the article

  • An Honest look at SharePoint Web Services

    - by juanlarios
    INTRODUCTION If you are a SharePoint developer you know that there are two basic ways to develop against SharePoint. 1) The object Model 2) Web services. SharePoint object model has the advantage of being quite rich. Anything you can do through the SharePoint UI as an administrator or end user, you can do through the object model. In fact everything that is done through the UI is done through the object model behind the scenes. The major disadvantage to getting at SharePoint this way is that the code needs to run on the server. This means that all web parts, event receivers, features, etc… all of this is code that is deployed to the server. The second way to get to SharePoint is through the built in web services. There are many articles on how to manipulate web services, how to authenticate to them and interact with them. The basic idea is that a remote application or process can contact SharePoint through a web service. Lots has been written about how great these web services are. This article is written to document the limitations, some of the issues and frustrations with working with SharePoint built in web services. Ultimately, for the tasks I was given to , SharePoint built in web services did not suffice. My evaluation of SharePoint built in services was compared against creating my own WCF Services to do what I needed. The current project I'm working on right now involved several "integration points". A remote application, installed on a separate server was to contact SharePoint and perform an task or operation. So I decided to start up Visual Studio and built a DLL and basically have 2 layers of logic. An integration layer and a data layer. A good friend of mine pointed me to SOLID principles and referred me to some videos and tutorials about it. I decided to implement the methodology (although a lot of the principles are common sense and I already incorporated in my coding practices). I was to deliver this dll to the application team and they would simply call the methods exposed by this dll and voila! it would do some task or operation in SharePoint. SOLUTION My integration layer implemented an interface that defined some of the basic integration tasks that I was to put together. My data layer was about the same, it implemented an interface with some of the tasks that I was going to develop. This gave me the opportunity to develop different data layers, ultimately different ways to get at SharePoint if I needed to. This is a classic SOLID principle. In this case it proved to be quite helpful because I wrote one data layer completely implementing SharePoint built in Web Services and another implementing my own WCF Service that I wrote. I should mention there is another layer underneath the data layer. In referencing SharePoint or WCF services in my visual studio project I created a class for every web service call. So for example, if I used List.asx. I created a class called "DocumentRetreival" this class would do the grunt work to connect to the correct URL, It would perform the basic operation of contacting the service and so on. If I used a view.asmx, I implemented a class called "ViewRetrieval" with the same idea as the last class but it would now interact with all he operations in view.asmx. This gave my data layer the ability to perform multiple calls without really worrying about some of the grunt work each class performs. This again, is a classic SOLID principle. So, in order to compare them side by side we can look at both data layers and with is involved in each. Lets take a look at the "Create Project" task or operation. The integration point is described as , "dll is to provide a way to create a project in SharePoint". Projects , in this case are basically document libraries. I am to implement a way in which a remote application can create a document library in SharePoint. Easy enough right? Use the list.asmx Web service in SharePoint. So here we go! Lets take a look at the code. I added the List.asmx web service reference to my project and this is the class that contacts it:  class DocumentRetrieval     {         private ListsSoapClient _service;      d   private bool _impersonation;         public DocumentRetrieval(bool impersonation, string endpt)         {             _service = new ListsSoapClient();             this.SetEndPoint(string.Format("{0}/{1}", endpt, ConfigurationManager.AppSettings["List"]));             _impersonation = impersonation;             if (_impersonation)             {                 _service.ClientCredentials.Windows.ClientCredential.Password = ConfigurationManager.AppSettings["password"];                 _service.ClientCredentials.Windows.ClientCredential.UserName = ConfigurationManager.AppSettings["username"];                 _service.ClientCredentials.Windows.AllowedImpersonationLevel =                     System.Security.Principal.TokenImpersonationLevel.Impersonation;             }     private void SetEndPoint(string p)          {             _service.Endpoint.Address = new EndpointAddress(p);          }          /// <summary>         /// Creates a document library with specific name and templateID         /// </summary>         /// <param name="listName">New list name</param>         /// <param name="templateID">Template ID</param>         /// <returns></returns>         public XmlElement CreateLibrary(string listName, int templateID, ref ExceptionContract exContract)         {             XmlDocument sample = new XmlDocument();             XmlElement viewCol = sample.CreateElement("Empty");             try             {                 _service.Open();                 viewCol = _service.AddList(listName, "", templateID);             }             catch (Exception ex)             {                 exContract = new ExceptionContract("DocumentRetrieval/CreateLibrary", ex.GetType(), "Connection Error", ex.StackTrace, ExceptionContract.ExceptionCode.error);                             }finally             {                 _service.Close();             }                                      return viewCol;         } } There was a lot more in this class (that I am not including) because i was reusing the grunt work and making other operations with LIst.asmx, For example, updating content types, changing or configuring lists or document libraries. One of the first things I noticed about working with the built in services is that you are really at the mercy of what is available to you. Before creating a document library (Project) I wanted to expose a IsProjectExisting method. This way the integration or data layer could recognize if a library already exists. Well there is no service call or method available to do that check. So this is what I wrote:   public bool DocLibExists(string listName, ref ExceptionContract exContract)         {             try             {                 var allLists = _service.GetListCollection();                                return allLists.ChildNodes.OfType<XmlElement>().ToList().Exists(x => x.Attributes["Title"].Value ==listName);             }             catch (Exception ex)             {                 exContract = new ExceptionContract("DocumentRetrieval/GetList/GetListWSCall", ex.GetType(), "Unable to Retrieve List Collection", ex.StackTrace, ExceptionContract.ExceptionCode.error);             }             return false;         } This really just gets an XMLElement with all the lists. It was then up to me to sift through the clutter and noise and see if Document library already existed. This took a little bit of getting used to. Now instead of working with code, you are working with XMLElement response format from web service. I wrote a LINQ query to go through and find if the attribute "Title" existed and had a value of the listname then it would return True, if not False. I didn't particularly like working this way. Dealing with XMLElement responses and then having to manipulate it to get at the exact data I was looking for. Once the check for the DocLibExists, was done, I would either create the document library or send back an error indicating the document library already existed. Now lets examine the code that actually creates the document library. It does what you are really after, it creates a document library. Notice how the template ID is really an integer. Every document library template in SharePoint has an ID associated with it. Document libraries, Image Library, Custom List, Project Tasks, etc… they all he a unique integer associated with it. Well, that's great but the client came back to me and gave me some specifics that each "project" or document library, should have. They specified they had 3 types of projects. Each project would have unique views, about 10 views for each project. Each Project specified unique configurations (auditing, versioning, content types, etc…) So what turned out to be a simple implementation of creating a document library as a repository for a project, turned out to be quite involved.  The first thing I thought of was to create a template for document library. There are other ways you can do this too. Using the web Service call, you could configure views, versioning, even content types, etc… the only catch is, you have to be working quite extensively with CAML. I am not fond of CAML. I can do it and work with it, I just don't like doing it. It is quite touchy and at times it is quite tough to understand where errors were made with CAML statements. Working with Web Services and CAML proved to be quite annoying. The service call would return a generic error message that did not particularly point me to a CAML statement syntax error, or even a CAML error. I was not sure if it was a security , performance or code based issue. It was quite tough to work with. At times it was difficult to work with because of the way SharePoint handles metadata. There are "Names", "Display Name", and "StaticName" fields. It was quite tough to understand at times, which one to use. So it took a lot of trial and error. There are tools that can help with CAML generation. There is also now intellisense for CAML statements in Visual Studio that might help but ultimately I'm not fond of CAML with Web Services.   So I decided on the template. So my plan was to create create a document library, configure it accordingly and then use The Template Builder that comes with the SharePoint SDK. This tool allows you to create site templates, list template etc… It is quite interesting because it does not generate an STP file, it actually generates an xml definition and a feature you can activate and make that template available on a site or site collection. The first issue I experienced with this is that one of the specifications to this template was that the "All Documents" view was to have 2 web parts on it. Well, it turns out that using the template builder , it did not include the web parts as part of the list template definition it generated. It backed up the settings, the views, the content types but not the custom web parts. I still decided to try this even without the web parts on the page. This new template defined a new Document library definition with a unique ID. The problem was that the service call accepts an int but it only has access to the built in library int definitions. Any new ones added or created will not be available to create. So this made it impossible for me to approach the problem this way.     I should also mention that one of the nice features about SharePoint is the ability to create list templates, back them up and then create lists based on that template. It can all be done by end user administrators. These templates are quite unique because they are saved as an STP file and not an xml definition. I also went this route and tried to see if there was another service call where I could create a document library based no given template name. Nope! none.      After some thinking I decide to implement a WCF service to do this creation for me. I was quite certain that the object model would allow me to create document libraries base on a template in which an ID was required and also templates saved as STP files. Now I don't want to bother with posting the code to contact WCF service because it's self explanatory, but I will post the code that I used to create a list with custom template. public ServiceResult CreateProject(string name, string templateName, string projectId)         {             string siteurl = SPContext.Current.Site.Url;             Guid webguid = SPContext.Current.Web.ID;                        using (SPSite site = new SPSite(siteurl))             {                 using (SPWeb rootweb = site.RootWeb)                 {                     SPListTemplateCollection temps = site.GetCustomListTemplates(rootweb);                     ProcessWeb(siteurl, webguid, web => Act_CreateProject(web, name, templateName, projectId, temps));                 }//SpWeb             }//SPSite              return _globalResult;                   }         private void Act_CreateProject(SPWeb targetsite, string name, string templateName, string projectId, SPListTemplateCollection temps) {                         var temp = temps.Cast<SPListTemplate>().FirstOrDefault(x => x.Name.Equals(templateName));             if (temp != null)             {                             try                 {                                         Guid listGuid = targetsite.Lists.Add(name, "", temp);                     SPList newList = targetsite.Lists[listGuid];                     _globalResult = new ServiceResult(true, "Success", "Success");                 }                 catch (Exception ex)                 {                     _globalResult = new ServiceResult(false, (string.IsNullOrEmpty(ex.Message) ? "None" : ex.Message + " " + templateName), ex.StackTrace.ToString());                 }                                       }        private void ProcessWeb(string siteurl, Guid webguid, Action<SPWeb> action) {                        using (SPSite sitecollection = new SPSite(siteurl)) {                 using (SPWeb web = sitecollection.AllWebs[webguid]) {                     action(web);                 }                     }                  } This code is actually some of the code I implemented for the service. there was a lot more I did on Project Creation which I will cover in my next blog post. I implemented an ACTION method to process the web. This allowed me to properly dispose the SPWEb and SPSite objects and not rewrite this code over and over again. So I implemented a WCF service to create projects for me, this allowed me to do a lot more than just create a document library with a template, it now gave me the flexibility to do just about anything the client wanted at project creation. Once this was implemented , the client came back to me and said, "we reference all our projects with ID's in our application. we want SharePoint to do the same". This has been something I have been doing for a little while now but I do hope that SharePoint 2010 can have more of an answer to this and address it properly. I have been adding metadata to SPWebs through property bag. I believe I have blogged about it before. This time it required metadata added to a document library. No problem!!! I also mentioned these web parts that were to go on the "All Documents" View. I took the opportunity to configure them to the appropriate settings. There were two settings that needed to be set on these web parts. One of them was a Project ID configured in the webpart properties. The following code enhances and replaces the "Act_CreateProject " method above:  private void Act_CreateProject(SPWeb targetsite, string name, string templateName, string projectId, SPListTemplateCollection temps) {                         var temp = temps.Cast<SPListTemplate>().FirstOrDefault(x => x.Name.Equals(templateName));             if (temp != null)             {                 SPLimitedWebPartManager wpmgr = null;                               try                 {                                         Guid listGuid = targetsite.Lists.Add(name, "", temp);                     SPList newList = targetsite.Lists[listGuid];                     SPFolder rootFolder = newList.RootFolder;                     rootFolder.Properties.Add(KEY, projectId);                     rootFolder.Update();                     if (rootFolder.ParentWeb != targetsite)                         rootFolder.ParentWeb.Dispose();                     if (!templateName.Contains("Natural"))                     {                         SPView alldocumentsview = newList.Views.Cast<SPView>().FirstOrDefault(x => x.Title.Equals(ALLDOCUMENTS));                         SPFile alldocfile = targetsite.GetFile(alldocumentsview.ServerRelativeUrl);                         wpmgr = alldocfile.GetLimitedWebPartManager(PersonalizationScope.Shared);                         ConfigureWebPart(wpmgr, projectId, CUSTOMWPNAME);                                              alldocfile.Update();                     }                                        if (newList.ParentWeb != targetsite)                         newList.ParentWeb.Dispose();                     _globalResult = new ServiceResult(true, "Success", "Success");                 }                 catch (Exception ex)                 {                     _globalResult = new ServiceResult(false, (string.IsNullOrEmpty(ex.Message) ? "None" : ex.Message + " " + templateName), ex.StackTrace.ToString());                 }                 finally                 {                     if (wpmgr != null)                     {                         wpmgr.Web.Dispose();                         wpmgr.Dispose();                     }                 }             }                         }       private void ConfigureWebPart(SPLimitedWebPartManager mgr, string prjId, string webpartname)         {             var wp = mgr.WebParts.Cast<System.Web.UI.WebControls.WebParts.WebPart>().FirstOrDefault(x => x.DisplayTitle.Equals(webpartname));             if (wp != null)             {                           (wp as ListRelationshipWebPart.ListRelationshipWebPart).ProjectID = prjId;                 mgr.SaveChanges(wp);             }         }   This Shows you how I was able to set metadata on the document library. It has to be added to the RootFolder of the document library, Unfortunately, the SPList does not have a Property bag that I can add a key\value pair to. It has to be done on the root folder. Now everything in the integration will reference projects by ID's and will not care about names. My, "DocLibExists" will now need to be changed because a web service is not set up to look at property bags.  I had to write another method on the Service to do the equivalent but with ID's instead of names.  The second thing you will notice about the code is the use of the Webpartmanager. I have seen several examples online, and also read a lot about memory leaks, The above code does not produce memory leaks. The web part manager creates an SPWeb, so just dispose it like I did. CONCLUSION This is a long long post so I will stop here for now, I will continue with more comparisons and limitations in my next post. My conclusion for this example is that Web Services will do the trick if you can suffer through CAML and if you are doing some simple operations. For Everything else, there's WCF! **** fireI apologize for the disorganization of this post, I was on a bus on a 12 hour trip to IOWA while I wrote it, I was half asleep and half awake, hopefully it makes enough sense to someone.

    Read the article

  • Advanced TSQL Tuning: Why Internals Knowledge Matters

    - by Paul White
    There is much more to query tuning than reducing logical reads and adding covering nonclustered indexes.  Query tuning is not complete as soon as the query returns results quickly in the development or test environments.  In production, your query will compete for memory, CPU, locks, I/O and other resources on the server.  Today’s entry looks at some tuning considerations that are often overlooked, and shows how deep internals knowledge can help you write better TSQL. As always, we’ll need some example data.  In fact, we are going to use three tables today, each of which is structured like this: Each table has 50,000 rows made up of an INTEGER id column and a padding column containing 3,999 characters in every row.  The only difference between the three tables is in the type of the padding column: the first table uses CHAR(3999), the second uses VARCHAR(MAX), and the third uses the deprecated TEXT type.  A script to create a database with the three tables and load the sample data follows: USE master; GO IF DB_ID('SortTest') IS NOT NULL DROP DATABASE SortTest; GO CREATE DATABASE SortTest COLLATE LATIN1_GENERAL_BIN; GO ALTER DATABASE SortTest MODIFY FILE ( NAME = 'SortTest', SIZE = 3GB, MAXSIZE = 3GB ); GO ALTER DATABASE SortTest MODIFY FILE ( NAME = 'SortTest_log', SIZE = 256MB, MAXSIZE = 1GB, FILEGROWTH = 128MB ); GO ALTER DATABASE SortTest SET ALLOW_SNAPSHOT_ISOLATION OFF ; ALTER DATABASE SortTest SET AUTO_CLOSE OFF ; ALTER DATABASE SortTest SET AUTO_CREATE_STATISTICS ON ; ALTER DATABASE SortTest SET AUTO_SHRINK OFF ; ALTER DATABASE SortTest SET AUTO_UPDATE_STATISTICS ON ; ALTER DATABASE SortTest SET AUTO_UPDATE_STATISTICS_ASYNC ON ; ALTER DATABASE SortTest SET PARAMETERIZATION SIMPLE ; ALTER DATABASE SortTest SET READ_COMMITTED_SNAPSHOT OFF ; ALTER DATABASE SortTest SET MULTI_USER ; ALTER DATABASE SortTest SET RECOVERY SIMPLE ; USE SortTest; GO CREATE TABLE dbo.TestCHAR ( id INTEGER IDENTITY (1,1) NOT NULL, padding CHAR(3999) NOT NULL,   CONSTRAINT [PK dbo.TestCHAR (id)] PRIMARY KEY CLUSTERED (id), ) ; CREATE TABLE dbo.TestMAX ( id INTEGER IDENTITY (1,1) NOT NULL, padding VARCHAR(MAX) NOT NULL,   CONSTRAINT [PK dbo.TestMAX (id)] PRIMARY KEY CLUSTERED (id), ) ; CREATE TABLE dbo.TestTEXT ( id INTEGER IDENTITY (1,1) NOT NULL, padding TEXT NOT NULL,   CONSTRAINT [PK dbo.TestTEXT (id)] PRIMARY KEY CLUSTERED (id), ) ; -- ============= -- Load TestCHAR (about 3s) -- ============= INSERT INTO dbo.TestCHAR WITH (TABLOCKX) ( padding ) SELECT padding = REPLICATE(CHAR(65 + (Data.n % 26)), 3999) FROM ( SELECT TOP (50000) n = ROW_NUMBER() OVER (ORDER BY (SELECT 0)) - 1 FROM master.sys.columns C1, master.sys.columns C2, master.sys.columns C3 ORDER BY n ASC ) AS Data ORDER BY Data.n ASC ; -- ============ -- Load TestMAX (about 3s) -- ============ INSERT INTO dbo.TestMAX WITH (TABLOCKX) ( padding ) SELECT CONVERT(VARCHAR(MAX), padding) FROM dbo.TestCHAR ORDER BY id ; -- ============= -- Load TestTEXT (about 5s) -- ============= INSERT INTO dbo.TestTEXT WITH (TABLOCKX) ( padding ) SELECT CONVERT(TEXT, padding) FROM dbo.TestCHAR ORDER BY id ; -- ========== -- Space used -- ========== -- EXECUTE sys.sp_spaceused @objname = 'dbo.TestCHAR'; EXECUTE sys.sp_spaceused @objname = 'dbo.TestMAX'; EXECUTE sys.sp_spaceused @objname = 'dbo.TestTEXT'; ; CHECKPOINT ; That takes around 15 seconds to run, and shows the space allocated to each table in its output: To illustrate the points I want to make today, the example task we are going to set ourselves is to return a random set of 150 rows from each table.  The basic shape of the test query is the same for each of the three test tables: SELECT TOP (150) T.id, T.padding FROM dbo.Test AS T ORDER BY NEWID() OPTION (MAXDOP 1) ; Test 1 – CHAR(3999) Running the template query shown above using the TestCHAR table as the target, we find that the query takes around 5 seconds to return its results.  This seems slow, considering that the table only has 50,000 rows.  Working on the assumption that generating a GUID for each row is a CPU-intensive operation, we might try enabling parallelism to see if that speeds up the response time.  Running the query again (but without the MAXDOP 1 hint) on a machine with eight logical processors, the query now takes 10 seconds to execute – twice as long as when run serially. Rather than attempting further guesses at the cause of the slowness, let’s go back to serial execution and add some monitoring.  The script below monitors STATISTICS IO output and the amount of tempdb used by the test query.  We will also run a Profiler trace to capture any warnings generated during query execution. DECLARE @read BIGINT, @write BIGINT ; SELECT @read = SUM(num_of_bytes_read), @write = SUM(num_of_bytes_written) FROM tempdb.sys.database_files AS DBF JOIN sys.dm_io_virtual_file_stats(2, NULL) AS FS ON FS.file_id = DBF.file_id WHERE DBF.type_desc = 'ROWS' ; SET STATISTICS IO ON ; SELECT TOP (150) TC.id, TC.padding FROM dbo.TestCHAR AS TC ORDER BY NEWID() OPTION (MAXDOP 1) ; SET STATISTICS IO OFF ; SELECT tempdb_read_MB = (SUM(num_of_bytes_read) - @read) / 1024. / 1024., tempdb_write_MB = (SUM(num_of_bytes_written) - @write) / 1024. / 1024., internal_use_MB = ( SELECT internal_objects_alloc_page_count / 128.0 FROM sys.dm_db_task_space_usage WHERE session_id = @@SPID ) FROM tempdb.sys.database_files AS DBF JOIN sys.dm_io_virtual_file_stats(2, NULL) AS FS ON FS.file_id = DBF.file_id WHERE DBF.type_desc = 'ROWS' ; Let’s take a closer look at the statistics and query plan generated from this: Following the flow of the data from right to left, we see the expected 50,000 rows emerging from the Clustered Index Scan, with a total estimated size of around 191MB.  The Compute Scalar adds a column containing a random GUID (generated from the NEWID() function call) for each row.  With this extra column in place, the size of the data arriving at the Sort operator is estimated to be 192MB. Sort is a blocking operator – it has to examine all of the rows on its input before it can produce its first row of output (the last row received might sort first).  This characteristic means that Sort requires a memory grant – memory allocated for the query’s use by SQL Server just before execution starts.  In this case, the Sort is the only memory-consuming operator in the plan, so it has access to the full 243MB (248,696KB) of memory reserved by SQL Server for this query execution. Notice that the memory grant is significantly larger than the expected size of the data to be sorted.  SQL Server uses a number of techniques to speed up sorting, some of which sacrifice size for comparison speed.  Sorts typically require a very large number of comparisons, so this is usually a very effective optimization.  One of the drawbacks is that it is not possible to exactly predict the sort space needed, as it depends on the data itself.  SQL Server takes an educated guess based on data types, sizes, and the number of rows expected, but the algorithm is not perfect. In spite of the large memory grant, the Profiler trace shows a Sort Warning event (indicating that the sort ran out of memory), and the tempdb usage monitor shows that 195MB of tempdb space was used – all of that for system use.  The 195MB represents physical write activity on tempdb, because SQL Server strictly enforces memory grants – a query cannot ‘cheat’ and effectively gain extra memory by spilling to tempdb pages that reside in memory.  Anyway, the key point here is that it takes a while to write 195MB to disk, and this is the main reason that the query takes 5 seconds overall. If you are wondering why using parallelism made the problem worse, consider that eight threads of execution result in eight concurrent partial sorts, each receiving one eighth of the memory grant.  The eight sorts all spilled to tempdb, resulting in inefficiencies as the spilled sorts competed for disk resources.  More importantly, there are specific problems at the point where the eight partial results are combined, but I’ll cover that in a future post. CHAR(3999) Performance Summary: 5 seconds elapsed time 243MB memory grant 195MB tempdb usage 192MB estimated sort set 25,043 logical reads Sort Warning Test 2 – VARCHAR(MAX) We’ll now run exactly the same test (with the additional monitoring) on the table using a VARCHAR(MAX) padding column: DECLARE @read BIGINT, @write BIGINT ; SELECT @read = SUM(num_of_bytes_read), @write = SUM(num_of_bytes_written) FROM tempdb.sys.database_files AS DBF JOIN sys.dm_io_virtual_file_stats(2, NULL) AS FS ON FS.file_id = DBF.file_id WHERE DBF.type_desc = 'ROWS' ; SET STATISTICS IO ON ; SELECT TOP (150) TM.id, TM.padding FROM dbo.TestMAX AS TM ORDER BY NEWID() OPTION (MAXDOP 1) ; SET STATISTICS IO OFF ; SELECT tempdb_read_MB = (SUM(num_of_bytes_read) - @read) / 1024. / 1024., tempdb_write_MB = (SUM(num_of_bytes_written) - @write) / 1024. / 1024., internal_use_MB = ( SELECT internal_objects_alloc_page_count / 128.0 FROM sys.dm_db_task_space_usage WHERE session_id = @@SPID ) FROM tempdb.sys.database_files AS DBF JOIN sys.dm_io_virtual_file_stats(2, NULL) AS FS ON FS.file_id = DBF.file_id WHERE DBF.type_desc = 'ROWS' ; This time the query takes around 8 seconds to complete (3 seconds longer than Test 1).  Notice that the estimated row and data sizes are very slightly larger, and the overall memory grant has also increased very slightly to 245MB.  The most marked difference is in the amount of tempdb space used – this query wrote almost 391MB of sort run data to the physical tempdb file.  Don’t draw any general conclusions about VARCHAR(MAX) versus CHAR from this – I chose the length of the data specifically to expose this edge case.  In most cases, VARCHAR(MAX) performs very similarly to CHAR – I just wanted to make test 2 a bit more exciting. MAX Performance Summary: 8 seconds elapsed time 245MB memory grant 391MB tempdb usage 193MB estimated sort set 25,043 logical reads Sort warning Test 3 – TEXT The same test again, but using the deprecated TEXT data type for the padding column: DECLARE @read BIGINT, @write BIGINT ; SELECT @read = SUM(num_of_bytes_read), @write = SUM(num_of_bytes_written) FROM tempdb.sys.database_files AS DBF JOIN sys.dm_io_virtual_file_stats(2, NULL) AS FS ON FS.file_id = DBF.file_id WHERE DBF.type_desc = 'ROWS' ; SET STATISTICS IO ON ; SELECT TOP (150) TT.id, TT.padding FROM dbo.TestTEXT AS TT ORDER BY NEWID() OPTION (MAXDOP 1, RECOMPILE) ; SET STATISTICS IO OFF ; SELECT tempdb_read_MB = (SUM(num_of_bytes_read) - @read) / 1024. / 1024., tempdb_write_MB = (SUM(num_of_bytes_written) - @write) / 1024. / 1024., internal_use_MB = ( SELECT internal_objects_alloc_page_count / 128.0 FROM sys.dm_db_task_space_usage WHERE session_id = @@SPID ) FROM tempdb.sys.database_files AS DBF JOIN sys.dm_io_virtual_file_stats(2, NULL) AS FS ON FS.file_id = DBF.file_id WHERE DBF.type_desc = 'ROWS' ; This time the query runs in 500ms.  If you look at the metrics we have been checking so far, it’s not hard to understand why: TEXT Performance Summary: 0.5 seconds elapsed time 9MB memory grant 5MB tempdb usage 5MB estimated sort set 207 logical reads 596 LOB logical reads Sort warning SQL Server’s memory grant algorithm still underestimates the memory needed to perform the sorting operation, but the size of the data to sort is so much smaller (5MB versus 193MB previously) that the spilled sort doesn’t matter very much.  Why is the data size so much smaller?  The query still produces the correct results – including the large amount of data held in the padding column – so what magic is being performed here? TEXT versus MAX Storage The answer lies in how columns of the TEXT data type are stored.  By default, TEXT data is stored off-row in separate LOB pages – which explains why this is the first query we have seen that records LOB logical reads in its STATISTICS IO output.  You may recall from my last post that LOB data leaves an in-row pointer to the separate storage structure holding the LOB data. SQL Server can see that the full LOB value is not required by the query plan until results are returned, so instead of passing the full LOB value down the plan from the Clustered Index Scan, it passes the small in-row structure instead.  SQL Server estimates that each row coming from the scan will be 79 bytes long – 11 bytes for row overhead, 4 bytes for the integer id column, and 64 bytes for the LOB pointer (in fact the pointer is rather smaller – usually 16 bytes – but the details of that don’t really matter right now). OK, so this query is much more efficient because it is sorting a very much smaller data set – SQL Server delays retrieving the LOB data itself until after the Sort starts producing its 150 rows.  The question that normally arises at this point is: Why doesn’t SQL Server use the same trick when the padding column is defined as VARCHAR(MAX)? The answer is connected with the fact that if the actual size of the VARCHAR(MAX) data is 8000 bytes or less, it is usually stored in-row in exactly the same way as for a VARCHAR(8000) column – MAX data only moves off-row into LOB storage when it exceeds 8000 bytes.  The default behaviour of the TEXT type is to be stored off-row by default, unless the ‘text in row’ table option is set suitably and there is room on the page.  There is an analogous (but opposite) setting to control the storage of MAX data – the ‘large value types out of row’ table option.  By enabling this option for a table, MAX data will be stored off-row (in a LOB structure) instead of in-row.  SQL Server Books Online has good coverage of both options in the topic In Row Data. The MAXOOR Table The essential difference, then, is that MAX defaults to in-row storage, and TEXT defaults to off-row (LOB) storage.  You might be thinking that we could get the same benefits seen for the TEXT data type by storing the VARCHAR(MAX) values off row – so let’s look at that option now.  This script creates a fourth table, with the VARCHAR(MAX) data stored off-row in LOB pages: CREATE TABLE dbo.TestMAXOOR ( id INTEGER IDENTITY (1,1) NOT NULL, padding VARCHAR(MAX) NOT NULL,   CONSTRAINT [PK dbo.TestMAXOOR (id)] PRIMARY KEY CLUSTERED (id), ) ; EXECUTE sys.sp_tableoption @TableNamePattern = N'dbo.TestMAXOOR', @OptionName = 'large value types out of row', @OptionValue = 'true' ; SELECT large_value_types_out_of_row FROM sys.tables WHERE [schema_id] = SCHEMA_ID(N'dbo') AND name = N'TestMAXOOR' ; INSERT INTO dbo.TestMAXOOR WITH (TABLOCKX) ( padding ) SELECT SPACE(0) FROM dbo.TestCHAR ORDER BY id ; UPDATE TM WITH (TABLOCK) SET padding.WRITE (TC.padding, NULL, NULL) FROM dbo.TestMAXOOR AS TM JOIN dbo.TestCHAR AS TC ON TC.id = TM.id ; EXECUTE sys.sp_spaceused @objname = 'dbo.TestMAXOOR' ; CHECKPOINT ; Test 4 – MAXOOR We can now re-run our test on the MAXOOR (MAX out of row) table: DECLARE @read BIGINT, @write BIGINT ; SELECT @read = SUM(num_of_bytes_read), @write = SUM(num_of_bytes_written) FROM tempdb.sys.database_files AS DBF JOIN sys.dm_io_virtual_file_stats(2, NULL) AS FS ON FS.file_id = DBF.file_id WHERE DBF.type_desc = 'ROWS' ; SET STATISTICS IO ON ; SELECT TOP (150) MO.id, MO.padding FROM dbo.TestMAXOOR AS MO ORDER BY NEWID() OPTION (MAXDOP 1, RECOMPILE) ; SET STATISTICS IO OFF ; SELECT tempdb_read_MB = (SUM(num_of_bytes_read) - @read) / 1024. / 1024., tempdb_write_MB = (SUM(num_of_bytes_written) - @write) / 1024. / 1024., internal_use_MB = ( SELECT internal_objects_alloc_page_count / 128.0 FROM sys.dm_db_task_space_usage WHERE session_id = @@SPID ) FROM tempdb.sys.database_files AS DBF JOIN sys.dm_io_virtual_file_stats(2, NULL) AS FS ON FS.file_id = DBF.file_id WHERE DBF.type_desc = 'ROWS' ; TEXT Performance Summary: 0.3 seconds elapsed time 245MB memory grant 0MB tempdb usage 193MB estimated sort set 207 logical reads 446 LOB logical reads No sort warning The query runs very quickly – slightly faster than Test 3, and without spilling the sort to tempdb (there is no sort warning in the trace, and the monitoring query shows zero tempdb usage by this query).  SQL Server is passing the in-row pointer structure down the plan and only looking up the LOB value on the output side of the sort. The Hidden Problem There is still a huge problem with this query though – it requires a 245MB memory grant.  No wonder the sort doesn’t spill to tempdb now – 245MB is about 20 times more memory than this query actually requires to sort 50,000 records containing LOB data pointers.  Notice that the estimated row and data sizes in the plan are the same as in test 2 (where the MAX data was stored in-row). The optimizer assumes that MAX data is stored in-row, regardless of the sp_tableoption setting ‘large value types out of row’.  Why?  Because this option is dynamic – changing it does not immediately force all MAX data in the table in-row or off-row, only when data is added or actually changed.  SQL Server does not keep statistics to show how much MAX or TEXT data is currently in-row, and how much is stored in LOB pages.  This is an annoying limitation, and one which I hope will be addressed in a future version of the product. So why should we worry about this?  Excessive memory grants reduce concurrency and may result in queries waiting on the RESOURCE_SEMAPHORE wait type while they wait for memory they do not need.  245MB is an awful lot of memory, especially on 32-bit versions where memory grants cannot use AWE-mapped memory.  Even on a 64-bit server with plenty of memory, do you really want a single query to consume 0.25GB of memory unnecessarily?  That’s 32,000 8KB pages that might be put to much better use. The Solution The answer is not to use the TEXT data type for the padding column.  That solution happens to have better performance characteristics for this specific query, but it still results in a spilled sort, and it is hard to recommend the use of a data type which is scheduled for removal.  I hope it is clear to you that the fundamental problem here is that SQL Server sorts the whole set arriving at a Sort operator.  Clearly, it is not efficient to sort the whole table in memory just to return 150 rows in a random order. The TEXT example was more efficient because it dramatically reduced the size of the set that needed to be sorted.  We can do the same thing by selecting 150 unique keys from the table at random (sorting by NEWID() for example) and only then retrieving the large padding column values for just the 150 rows we need.  The following script implements that idea for all four tables: SET STATISTICS IO ON ; WITH TestTable AS ( SELECT * FROM dbo.TestCHAR ), TopKeys AS ( SELECT TOP (150) id FROM TestTable ORDER BY NEWID() ) SELECT T1.id, T1.padding FROM TestTable AS T1 WHERE T1.id = ANY (SELECT id FROM TopKeys) OPTION (MAXDOP 1) ; WITH TestTable AS ( SELECT * FROM dbo.TestMAX ), TopKeys AS ( SELECT TOP (150) id FROM TestTable ORDER BY NEWID() ) SELECT T1.id, T1.padding FROM TestTable AS T1 WHERE T1.id IN (SELECT id FROM TopKeys) OPTION (MAXDOP 1) ; WITH TestTable AS ( SELECT * FROM dbo.TestTEXT ), TopKeys AS ( SELECT TOP (150) id FROM TestTable ORDER BY NEWID() ) SELECT T1.id, T1.padding FROM TestTable AS T1 WHERE T1.id IN (SELECT id FROM TopKeys) OPTION (MAXDOP 1) ; WITH TestTable AS ( SELECT * FROM dbo.TestMAXOOR ), TopKeys AS ( SELECT TOP (150) id FROM TestTable ORDER BY NEWID() ) SELECT T1.id, T1.padding FROM TestTable AS T1 WHERE T1.id IN (SELECT id FROM TopKeys) OPTION (MAXDOP 1) ; SET STATISTICS IO OFF ; All four queries now return results in much less than a second, with memory grants between 6 and 12MB, and without spilling to tempdb.  The small remaining inefficiency is in reading the id column values from the clustered primary key index.  As a clustered index, it contains all the in-row data at its leaf.  The CHAR and VARCHAR(MAX) tables store the padding column in-row, so id values are separated by a 3999-character column, plus row overhead.  The TEXT and MAXOOR tables store the padding values off-row, so id values in the clustered index leaf are separated by the much-smaller off-row pointer structure.  This difference is reflected in the number of logical page reads performed by the four queries: Table 'TestCHAR' logical reads 25511 lob logical reads 000 Table 'TestMAX'. logical reads 25511 lob logical reads 000 Table 'TestTEXT' logical reads 00412 lob logical reads 597 Table 'TestMAXOOR' logical reads 00413 lob logical reads 446 We can increase the density of the id values by creating a separate nonclustered index on the id column only.  This is the same key as the clustered index, of course, but the nonclustered index will not include the rest of the in-row column data. CREATE UNIQUE NONCLUSTERED INDEX uq1 ON dbo.TestCHAR (id); CREATE UNIQUE NONCLUSTERED INDEX uq1 ON dbo.TestMAX (id); CREATE UNIQUE NONCLUSTERED INDEX uq1 ON dbo.TestTEXT (id); CREATE UNIQUE NONCLUSTERED INDEX uq1 ON dbo.TestMAXOOR (id); The four queries can now use the very dense nonclustered index to quickly scan the id values, sort them by NEWID(), select the 150 ids we want, and then look up the padding data.  The logical reads with the new indexes in place are: Table 'TestCHAR' logical reads 835 lob logical reads 0 Table 'TestMAX' logical reads 835 lob logical reads 0 Table 'TestTEXT' logical reads 686 lob logical reads 597 Table 'TestMAXOOR' logical reads 686 lob logical reads 448 With the new index, all four queries use the same query plan (click to enlarge): Performance Summary: 0.3 seconds elapsed time 6MB memory grant 0MB tempdb usage 1MB sort set 835 logical reads (CHAR, MAX) 686 logical reads (TEXT, MAXOOR) 597 LOB logical reads (TEXT) 448 LOB logical reads (MAXOOR) No sort warning I’ll leave it as an exercise for the reader to work out why trying to eliminate the Key Lookup by adding the padding column to the new nonclustered indexes would be a daft idea Conclusion This post is not about tuning queries that access columns containing big strings.  It isn’t about the internal differences between TEXT and MAX data types either.  It isn’t even about the cool use of UPDATE .WRITE used in the MAXOOR table load.  No, this post is about something else: Many developers might not have tuned our starting example query at all – 5 seconds isn’t that bad, and the original query plan looks reasonable at first glance.  Perhaps the NEWID() function would have been blamed for ‘just being slow’ – who knows.  5 seconds isn’t awful – unless your users expect sub-second responses – but using 250MB of memory and writing 200MB to tempdb certainly is!  If ten sessions ran that query at the same time in production that’s 2.5GB of memory usage and 2GB hitting tempdb.  Of course, not all queries can be rewritten to avoid large memory grants and sort spills using the key-lookup technique in this post, but that’s not the point either. The point of this post is that a basic understanding of execution plans is not enough.  Tuning for logical reads and adding covering indexes is not enough.  If you want to produce high-quality, scalable TSQL that won’t get you paged as soon as it hits production, you need a deep understanding of execution plans, and as much accurate, deep knowledge about SQL Server as you can lay your hands on.  The advanced database developer has a wide range of tools to use in writing queries that perform well in a range of circumstances. By the way, the examples in this post were written for SQL Server 2008.  They will run on 2005 and demonstrate the same principles, but you won’t get the same figures I did because 2005 had a rather nasty bug in the Top N Sort operator.  Fair warning: if you do decide to run the scripts on a 2005 instance (particularly the parallel query) do it before you head out for lunch… This post is dedicated to the people of Christchurch, New Zealand. © 2011 Paul White email: @[email protected] twitter: @SQL_Kiwi

    Read the article

  • Connection Reset on MySQL query

    - by sunwukung
    OK, I'm flummoxed.(i've asked this question over on Stack too - but I need to get it fixed so I'm asking here too - any help is GREATLY appreciated) I'm trying to execute a query on a database (locally) and I keep getting a connection reset error. I've been using the method below in a generic DAO class to build a query string and pass to Zend_Db API. public function insert($params) { $loop = false; $keys = $values = ''; foreach($params as $k => $v){ if($loop == true){ $keys .= ','; $values .= ','; } $keys .= $this->db->quoteIdentifier($k); $values .= $this->db->quote($v); $loop = true; } $sql = "INSERT INTO " . $this->table_name . " ($keys) VALUES ($values)"; //formatResult returns an array of info regarding the status and any result sets of the query //I've commented that method call out anyway, so I don't think it's that try { $this->db->query($sql); return $this->formatResult(array( true, 'New record inserted into: '.$this->table_name )); }catch(PDOException $e) { return $this->formatResult($e); } } So far, this has worked fine - the errors have been occurring since we generated new tables to record user input. The insert string looks like this: INSERT INTO tablename(`id`,`title`,`summary`,`description`,`keywords`,`type_id`,`categories`) VALUES ('5539','Sample Title','Sample content',' \'Lorem ipsum dolor sit amet, consectetur adipiscing elit. In et pellentesque mauris. Curabitur hendrerit, leo id ultrices pellentesque, est purus mattis ligula, vitae imperdiet neque ligula bibendum sapien. Curabitur aliquet nisi et odio pharetra tincidunt. Phasellus sed iaculis nisl. Fusce commodo mauris et purus vehicula dictum. Nulla feugiat molestie accumsan. Donec fermentum libero in risus tempus elementum aliquam et magna. Fusce vitae sem metus. Aenean commodo pharetra risus, nec pellentesque augue ullamcorper nec. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nullam vel elit libero. Vestibulum in turpis nunc.\'','this,is,a,sample,array',1,'category title') Here are the parameters it's getting before assembling the query (var_dump): array 'id' => string '1' (length=4) 'title' => string 'Sample Title' (length=12) 'summary' => string 'Sample content' (length=14) 'description' => string '<p>'Lorem ipsum dolor sit amet, consectetur adipiscing elit. In et pellentesque mauris. Curabitur hendrerit, leo id ultrices pellentesque, est purus mattis ligula, vitae imperdiet neque ligula bibendum sapien. Curabitur aliquet nisi et odio pharetra tincidunt. Phasellus sed iaculis nisl. Fusce commodo mauris et purus vehicula dictum. Nulla feugiat molestie accumsan. Donec fermentum libero in risus tempus elementum aliquam et magna. Fusce vitae sem metus. Aenean commodo pharetra risus, nec pellentesque augue'... (length=677) 'keywords' => string 'this,is,a,sample,array' (length=22) 'type_id' => int 1 'categories' => string 'category title' (length=43) The next port of call was checking the limits on the table, since it seems to insert if the length of "description" is around the 300 mark (it varies between 310 - 330). The field limit is set to VARCHAR(1500) and the validation on this field won't allow anything past bigger than 1200 with HTML, 800 without. The real kicker is that if I take this sql string and execute it via the command line, it works fine - so I can't for the life of me figure out what's wrong. I've tried extending the server parameters i.e. http://stackoverflow.com/questions/1964554/unexpected-connection-reset-a-php-or-an-apache-issue So, in a nutshell, I'm stumped. Any ideas?

    Read the article

  • Splitting a raidctl mirror safely

    - by milkfilk
    I have a Sun T5220 server with the onboard LSI card and two disks that were in a RAID 1 mirror. The data is not important right now but we had a failed disk and are trying to understand how to do this for real if we had to recover from a failure. The initial situation looked like this: # raidctl -l c1t0d0 Volume Size Stripe Status Cache RAID Sub Size Level Disk ---------------------------------------------------------------- c1t0d0 136.6G N/A DEGRADED OFF RAID1 0.1.0 136.6G GOOD N/A 136.6G FAILED Green light on the 0.0.0 disk. Find / lights up the 0.1.0 disk. So I know I have a bad drive and which one it is. Server still boots obviously. First, we tried putting a new disk in. This disk came from an unknown source. Format would not see it, cfgadm -al would not see it so raidctl -l would not see it. I figure it's bad. We tried another disk from another spare server: # raidctl -c c1t1d0 c1t0d0 (where t1 is my good disk - 0.1.0) Disk has occupied space. Also the different syntax options don't change anything: # raidctl -C "0.1.0 0.0.0" -r 1 1 Disk has occupied space. # raidctl -C "0.1.0 0.0.0" 1 Disk has occupied space. Ok. Maybe this is because the disk from the spare server had a RAID 1 on it already. Aha, I can see another volume in raidctl: # raidctl -l Controller: 1 Volume:c1t1d0 (this is my server's root mirror) Volume:c1t132d0 (this is the foreign root mirror) Disk: 0.0.0 Disk: 0.1.0 ... No problem. I don't care about the data, I'll just delete the foreign mirror. # raidctl -d c1t132d0 (warning about data deletion but it works) At this point, /usr/bin/ binaries freak out. By that I mean, ls -l /usr/bin/which shows 1.4k but cat /usr/bin/which gives me a newline. Great, I just blew away the binaries (ie: binaries in mem still work)? I bounce the box. It all comes back fine. WTF. Anyway, back to recreating my mirror. # raidctl -l Controller: 1 Volume:c1t1d0 (this is my server's root mirror) Disk: 0.0.0 Disk: 0.1.0 ... Man says that you can delete a mirror and it will split it. Ok, I'll delete the root mirror. # raidctl -d c1t0d0 Array in use. (this might not be the exact error) I googled this and found of course you can't do this (even with -f) while booted off the mirror. Ok. I boot cdrom -s and deleted the volume. Now I have one disk that has a type of "LSI-Logical-Volume" on c1t1d0 (where my data is) and a brand new "Hitachi 146GB" on c1t0d0 (what I'm trying to mirror to): (booted off the CD) # raidctl -c c1t1d0 c1t0d0 (man says it's source destination for mirroring) Illegal Array Layout. # raidctl -C "0.1.0 0.0.0" -r 1 1 (alt syntax per man) Illegal Array Layout. # raidctl -C "0.1.0 0.0.0" 1 (assumes raid1, no help) Illegal Array Layout. Same size disks, same manufacturer but I did delete the volume instead of throwing in a blank disk and waiting for it to resync. Maybe this was a critical error. I tried selecting the type in format for my good disk to be a plain 146gb disk but it resets the partition table which I'm pretty sure would wipe the data (bad if this was production). Am I boned? Anyone have experience with breaking and resyncing a mirror? There's nothing on Google about "Illegal Array Layout" so here's my contrib to the search gods.

    Read the article

  • How to troubleshoot a PHP script that causes a Segmenation Fault?

    - by johnlai2004
    I posted this on stackoverflow.com as well because I'm not sure if this is a programming problem or a server problem. I'm using ubuntu 9.10, apache2, mysql5 and php5. I've noticed an unusual problem with some of my php programs. Sometimes when visiting a page like profile.edit.php, the browser throws a dialogue box asking to download profile.edit.php page. When I download it, there's nothing in the file. profile.edit.php is supposed to be a web form that edits user information. I've noticed this on some of my other php pages as well. I look in my apache error logs, and I see a segmentation fault message: [Mon Mar 08 15:40:10 2010] [notice] child pid 480 exit signal Segmentation fault (11) And also, the issue may or may not appear depending on which server I deploy my application too. Additonal Details This doesn't happen all the time though. It only happens sometimes. For example, profile.edit.php will load properly. But as soon as I hit the save button (form action="profile.edit.php?save=true"), then the page asks me to download profile.edit.php. Could it be that sometimes my php scripts consume too much resources? Sample code Upon save action, my profile.edit.php includes a data_access_object.php file. I traced the code in data_access_object.php to this line here if($params[$this->primaryKey]) { $q = "UPDATE $this->tableName SET ".implode(', ', $fields)." WHERE ".$this->primaryKey." = ?$this->primaryKey"; $this->bind($this->primaryKey, $params[$this->primaryKey], $this->tblFields[$this->primaryKey]['mysqlitype']); } else { $q = "INSERT $this->tableName SET ".implode(', ', $fields); } // Code executes perfectly up to this point // echo 'print this'; exit; // if i uncomment this line, profile.edit.php will actually show 'print this'. If I leave it commented, the browser will ask me to download profile.edit.php if(!$this->execute($q)){ $this->errorSave = -3; return false;} // When I jumped into the function execute(), every line executed as expected, right up to the return statement. And if it helps, here's the function execute($sql) in data_access_object.php function execute($sql) { // find all list types and explode them // eg. turn ?listId into ?listId0,?listId1,?listId2 $arrListParam = array_bubble_up('arrayName', $this->arrBind); foreach($arrListParam as $listName) if($listName) { $explodeParam = array(); $arrList = $this->arrBind[$listName]['value']; foreach($arrList as $key=>$val) { $newParamName = $listName.$key; $this->bind($newParamName,$val,$this->arrBind[$listName]['type']); $explodeParam[] = '?'.$newParamName; } $sql = str_replace("?$listName", implode(',',$explodeParam), $sql); } // replace all ?varName with ? for syntax compliance $sqlParsed = preg_replace('/\?[\w\d_\.]+/', '?', $sql); $this->stmt->prepare($sqlParsed); // grab all the parameters from the sql to create bind conditions preg_match_all('/\?[\w\d_\.]+/', $sql, $matches); $matches = $matches[0]; // store bind conditions $types = ''; $params = array(); foreach($matches as $paramName) { $types .= $this->arrBind[str_replace('?', '', $paramName)]['type']; $params[] = $this->arrBind[str_replace('?', '', $paramName)]['value']; } $input = array('types'=>$types) + $params; // bind it if(!empty($types)) call_user_func_array(array($this->stmt, 'bind_param'), $input); $stat = $this->stmt->execute(); if($GLOBALS['DEBUG_SQL']) echo '<p style="font-weight:bold;">SQL error after execution:</p> ' . $this->stmt->error.'<p>&nbsp;</p>'; $this->arrBind = array(); return $stat; }

    Read the article

  • Intel Rapid Storage / Smart Response SSD caching issue

    - by goober
    Background Recently built my own PC. It works! Almost. It's been a while since getting into the guts of these things, so I'm familiar but may be missing something simple. FYI, I don't care about blowing the OS away -- it's brand new and we can go back from scratch as many times as necessary. Goal / Issue I'd like to use the SSD to take advantage of Intel's Smart Response technology (allows the SSD to act as a cache for HDDs) I would like the SSD cache to act as a cache for my HDDs, which I would like to be in a RAID1 array (so I get the speed from the SSD and the redundancy from the RAID1) However, Windows only sees the drive in device manager (not as a drive), so I'm unsure what to do about that. Related: as far as I know, for this to work, the drives all have to be in a single RAID array (i.e. a RAID0 pairing of the SSD and the RAID1 HDD array). However, when attempting this at the BIOS level, I am told there is not enough space for an array. Steps so Far Moved the SSD onto the Intel controller (I'd had it on the Marvel 6.0 controller instead of the Intel controller, so the BIOS was only seeing it in a strange way) Updated the BIOS of the motherboard to the latest version Reinstalled Intel's RST (iRST?) software several times, as some forums reported it working after reinstalling 3 times (which does not inspire confidence). Checked Intel storage: it does see the SSD as a physical, non-RAID device. However, it says no space exists if I try to create an array. Checked the BIOS: it does not show up in the boot order, but is an option that can be selected under boot options. Tried the firmware update for that model. Issue: the firmware CD doesn't detect a drive; maybe the Intel storage controller is making it difficult? moved the ssd to the marvel controller. The firmware update cd appeared to hang while searching for drives. swapped out the SATA cable for the manufacturer's and moved back to the intel storage controller. Noticed at this point that in the Intel RST software, a device DOES show up in addition to the RAID set -- only shown as a "60 GB internal disk". Windows doesn't appear to see it as a drive, but it does still show in device manager. Move SSD to port from 0-3 on MOBO and set SATA mode to IDE (after disconnecting RAID1 config) to allow the firmware update to work. Firmware was already at the latest version. Next Steps ? Components involved ASUS P8Z68-V PRO motherboard (Intel Z68 Chipset) Intel i7 2600k Processor 2 x 1TB 7200 RPM HDDs 64 GB Crucial M4 SSD (M4-CT064M4SSD2) For Reference -- Storage Configuration Intel 3 gbps Intel 3gbps Intel 6gbps Marvel 6gbps +----------+ +----------+ +----------+ +----------+ | | <----+ | | +-+ | | | |----------| | |----------| |-|--------| |----------| | | | | + | | | | | | +----------+ | +--|-------+ +-|--------+ +----------+ | | | + v v | 1 TB HDD 64 GB SSD + +> 1 TB HDD For Reference -- Intel RST (v10.8.0.1003) Screenshot Don't mind the "rebuilding" -- knocked a power cable out at one point; it's doing its job, not an indicator of a bad HDD. Any thoughts? Thanks in advance for any help!

    Read the article

< Previous Page | 255 256 257 258 259 260 261 262 263 264 265 266  | Next Page >