Search Results

Search found 17124 results on 685 pages for 'final cut pro'.

Page 331/685 | < Previous Page | 327 328 329 330 331 332 333 334 335 336 337 338  | Next Page >

  • Servlet response wrapper has encoding problem

    - by John O
    A servlet response wrapper is being used in a Servlet Filter. The idea is that the response is manipulated, with a 'nonce' value being injected into forms, as part of defence against CSRF attacks. The web app is using UTF-8 everywhere. When the Servlet Filter is absent, no problems. When the filter is added, encoding issues occur. (It seems as if the response is reverting to 8859-1.) The guts of the code : final class CsrfResponseWrapper extends AbstractResponseWrapper { ... byte[] modifyResponse(byte[] aInputResponse){ ... String originalInput = new String(aInputResponse, encoding); String modifiedResult = addHiddenParamToPostedForms(originalInput); result = modifiedResult.getBytes(encoding); ... } ... } As I understand it, the transition between byte-land and String-land should specify an encoding. That is done here, as you can see, in two places. The value of the 'encoding' variable is 'UTF-8'; the alteration of the String itself is standard string manipulation (with a regex), and never specifies an encoding (addHiddenParamToPostedForms). Where am I in error about the encoding? EDIT: Here is the base class (sorry it's rather long): package hirondelle.web4j.security; import javax.servlet.ServletOutputStream; import javax.servlet.ServletResponse; import javax.servlet.http.HttpServletResponse; import javax.servlet.http.HttpServletResponseWrapper; import java.io.ByteArrayOutputStream; import java.io.IOException; import java.io.PrintWriter; /** Abstract Base Class for altering response content. (May be useful in future contexts as well. For now, keep package-private.) */ abstract class AbstractResponseWrapper extends HttpServletResponseWrapper { AbstractResponseWrapper(ServletResponse aServletResponse) throws IOException { super((HttpServletResponse)aServletResponse); fOutputStream = new ModifiedOutputStream(aServletResponse.getOutputStream()); fWriter = new PrintWriter(fOutputStream); } /** Return the modified response. */ abstract byte[] modifyResponse(byte[] aInputResponse); /** Standard servlet method. */ public final ServletOutputStream getOutputStream() { //fLogger.fine("Modified Response : Getting output stream."); if ( fWriterReturned ) { throw new IllegalStateException(); } fOutputStreamReturned = true; return fOutputStream; } /** Standard servlet method. */ public final PrintWriter getWriter() { //fLogger.fine("Modified Response : Getting writer."); if ( fOutputStreamReturned ) { throw new IllegalStateException(); } fWriterReturned = true; return fWriter; } // PRIVATE /* Well-behaved servlets return either an OutputStream or a PrintWriter, but not both. */ private PrintWriter fWriter; private ModifiedOutputStream fOutputStream; /* These items are used to implement conformance to the javadoc for ServletResponse, regarding exceptions being thrown. */ private boolean fWriterReturned; private boolean fOutputStreamReturned; /** Modified low level output stream. */ private class ModifiedOutputStream extends ServletOutputStream { public ModifiedOutputStream(ServletOutputStream aOutputStream) { fServletOutputStream = aOutputStream; fBuffer = new ByteArrayOutputStream(); } /** Must be implemented to make this class concrete. */ public void write(int aByte) { fBuffer.write(aByte); } public void close() throws IOException { if ( !fIsClosed ){ processStream(); fServletOutputStream.close(); fIsClosed = true; } } public void flush() throws IOException { if ( fBuffer.size() != 0 ){ if ( !fIsClosed ) { processStream(); fBuffer = new ByteArrayOutputStream(); } } } /** Perform the core processing, by calling the abstract method. */ public void processStream() throws IOException { fServletOutputStream.write(modifyResponse(fBuffer.toByteArray())); fServletOutputStream.flush(); } // PRIVATE // private ServletOutputStream fServletOutputStream; private ByteArrayOutputStream fBuffer; /** Tracks if this stream has been closed. */ private boolean fIsClosed = false; } }

    Read the article

  • java will this threading setup work or what can i be doing wrong

    - by Erik
    Im a bit unsure and have to get advice. I have the: public class MyApp extends JFrame{ And from there i do; MyServer = new MyServer (this); MyServer.execute(); MyServer is a: public class MyServer extends SwingWorker<String, Object> { MyServer is doing listen_socket.accept() in the doInBackground() and on connection it create a new class Connection implements Runnable { I have the belove DbHelper that are a singleton. It holds an Sqlite connected. Im initiating it in the above MyApp and passing references all the way in to my runnable: class Connection implements Runnable { My question is what will happen if there are two simultaneous read or `write? My thought here was the all methods in the singleton are synchronized and would put all calls in the queue waiting to get a lock on the synchronized method. Will this work or what can i change? public final class DbHelper { private boolean initalized = false; private String HomePath = ""; private File DBFile; private static final String SYSTEM_TABLE = "systemtable"; Connection con = null; private Statement stmt; private static final ContentProviderHelper instance = new ContentProviderHelper (); public static ContentProviderHelper getInstance() { return instance; } private DbHelper () { if (!initalized) { initDB(); initalized = true; } } private void initDB() { DBFile = locateDBFile(); try { Class.forName("org.sqlite.JDBC"); // create a database connection con = DriverManager.getConnection("jdbc:sqlite:J:/workspace/workComputer/user_ptpp"); } catch (SQLException e) { e.printStackTrace(); } catch (ClassNotFoundException e) { e.printStackTrace(); } } private File locateDBFile() { File f = null; try{ HomePath = System.getProperty("user.dir"); System.out.println("HomePath: " + HomePath); f = new File(HomePath + "/user_ptpp"); if (f.canRead()) return f; else { boolean success = f.createNewFile(); if (success) { System.out.println("File did not exist and was created " + HomePath); // File did not exist and was created } else { System.out.println("File already exists " + HomePath); // File already exists } } } catch (IOException e) { System.out.println("Maybe try a new directory. " + HomePath); //Maybe try a new directory. } return f; } public String getHomePath() { return HomePath; } private synchronized String getDate(){ SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); Date date = new Date(); return dateFormat.format(date); } public synchronized String getSelectedSystemTableColumn( String column) { String query = "select "+ column + " from " + SYSTEM_TABLE ; try { stmt = con.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY); ResultSet rs = stmt.executeQuery(query); while (rs.next()) { String value = rs.getString(column); if(value == null || value == "") return ""; else return value; } } catch (SQLException e ) { e.printStackTrace(); return ""; } finally { } return ""; } }

    Read the article

  • Scratch disks on solid state drives

    - by Kato
    For something like Final Cut Pro where you have scratch disks, is it absolutely a bad idea to use a solid state drive? There would be a lot of writing, but I'm thinking it would be less for video editing then say, programming? The read/write cycles for SSDs still seem pretty long...

    Read the article

  • Can someone explain this color wheel code to me?

    - by user1869438
    I just started doing java and i need some help with understanding this code. I got it from a this website. This is supposed to be code for a color wheel but i don't really understand how it works, especially the final ints STEPS and SLICES. import java.awt.Color; import objectdraw.*; public class ColorWheel extends WindowController { private double brightness; private Text text; private FilledRect swatch; private Location center; private int size; private FilledRect brightnessOverlay; private static final int SLICES = 96; private static final int STEPS = 16; public void begin() { canvas.setBackground(Color.BLACK); brightness = 1.; size = Math.min(canvas.getWidth(), canvas.getHeight() - 20); center = new Location(canvas.getWidth() / 2, size / 2); for(int j = STEPS; j >= 1; j--) { int arcSize = size * j / STEPS; int x = center.getX() - arcSize / 2; int y = center.getY() - arcSize / 2; for(int i = 0; i < SLICES; i++) { Color c = Color.getHSBColor((float)i / SLICES, (float)j / STEPS, (float)brightness); new FilledArc(x, y, arcSize, arcSize, i * 360. / SLICES, 360. / SLICES + .5, c, canvas); } } swatch = new FilledRect(0, canvas.getHeight() - 20, canvas.getWidth(), 20, Color.BLACK, canvas); brightnessOverlay = new FilledRect(0, 0, canvas.getWidth(), canvas.getHeight() - 20, new Color(0, 0, 0, 0), canvas); text = new Text("", canvas.getWidth() / 2, canvas.getHeight() - 18, canvas); text.setAlignment(Text.CENTER, Text.TOP); text.setBold(true); } public void onMouseDrag(Location point) { brightness = (canvas.getHeight() - point.getY()) / (double)(canvas.getHeight()); if(brightness < 0) { brightness = 0; } else if(brightness > 1) { brightness = 1; } if(brightness < .5) { text.setColor(Color.WHITE); } else { text.setColor(Color.BLACK); } brightnessOverlay.setColor(new Color(0f, 0f, 0f, (float)(1 - brightness))); } public void onMouseMove(Location point) { double saturation = 2 * center.distanceTo(point) / size; if(saturation > 1) { text.setText(""); swatch.setColor(Color.BLACK); return; } double hue = -Math.atan2(point.getY() - center.getY(), point.getX() - center.getX()) / (2 * Math.PI); if(hue < 0) { hue += 1; } swatch.setColor(Color.getHSBColor((float)hue, (float)saturation, (float)brightness)); text.setText("Color.getHSBColor(" + Text.formatDecimal(hue, 2) + "f, " + Text.formatDecimal(saturation, 2) + "f, " + Text.formatDecimal(brightness, 2) + "f)"); } }

    Read the article

  • How to setup RAID partitions with parted?

    - by psycketom
    I'm going through the https://wiki.archlinux.org/index.php/RAID guide in here, but I'm stuck on Partition Tables. Since my drives are 3TB, fdisk and cfdisk won't cut it due to their 2GB limit, but they are straight forward when managing partitions - adding da or fd as types. But, there is not that straight forward guide for RAID partition setup with parted. So, how do I make Non-FS or RAID partition with parted?

    Read the article

  • center menu within the 960 grid

    - by Kyle Monti
    I have been working on 960 grid,(http://960.gs/) and I used an old style menu i've used in the past from a few years ago and for some reason with the 960 grid, the menu is floating left and I want it centered. ul#menu { width:940px; height:61px; background: url(../images/menu_bg.png) no-repeat; list-style:none; padding-top:0; padding-left:0; margin: 0; } ul#menu li { float:left; } ul#menu li a { background: url(../images/menu_splice_color.png) no-repeat scroll top left; display:block; height:61px; position:relative; } ul#menu li a.shel { width:135px; } ul#menu li a.golf { width:84px; background-position:-135px 0px; } ul#menu li a.pro { width:119px; background-position:-219px 0px; } ul#menu li a.event { width:94px; background-position:-338px 0px; } ul#menu li a.member { width:148px; background-position:-432px 0px; } ul#menu li a.bistro { width:91px; background-position:-580px 0px; } ul#menu li a.contact { width:115px; background-position:-671px 0px; } ul#menu li a span { background: url(../images/menu_splice_color.png) no-repeat scroll bottom left; display:block; position:absolute; top:0; left:0px; height:100%; width:100%; z-index:100; } ul#menu li a.shel span { background-position:0px -61px; } ul#menu li a.golf span { background-position:-135px -61px; } ul#menu li a.pro span { background-position:-219px -61px; } ul#menu li a.events span { background-position:-338px -61px; } ul#menu li a.member span { background-position:-432px -61px; } ul#menu li a.bistro span { background-position:-580px -61px; } ul#menu li a.contact span { background-position:-672px -61px; } and my generic html markup is <div class="container_16"> <!-- Navigation Start --> <div class="grid_16"> <ul id="menu"> <li><a href="#" class="shel"><span></span></a></li> <li><a href="#" class="golf"><span></span></a></li> <li><a href="#" class="pro"><span></span></a></li> <li><a href="#" class="events"><span></span></a></li> <li><a href="#" class="member"><span></span></a></li> <li><a href="#" class="bistro"><span></span></a></li> <li><a href="#" class="contact"><span></span></a></li> </ul> </div> <div class="clear"></div> </div> And I use jquery animations to roll over the images. $(function() { $("ul#menu span").css("opacity","0"); $("ul#menu span").hover(function () { $(this).stop().animate({ opacity: 1 }, "slow"); }, function () { $(this).stop().animate({ opacity: 0 }, "slow"); }); });

    Read the article

  • Using my custom colormap in Java for images

    - by John
    Hi everyone! I've got a question concering a colormapping via index. I tried this code found on http://www.podgoretsky.pri.ee/ftp/Docs/Java/Tricks%20of%20the%20Java%20Programming%20Gurus/ch12.htm // Gradient.java // Imports import java.applet.Applet; import java.awt.; import java.awt.image.; public class Gradient extends Applet { final int colors = 32; final int width = 200; final int height = 200; Image img; public void init() { // Create the color map byte[] rbmap = new byte[colors]; byte[] gmap = new byte[colors]; for (int i = 0; i < colors; i++) gmap[i] = (byte)((i * 255) / (colors - 1)); // Create the color model int bits = (int)Math.ceil(Math.log(colors) / Math.log(2)); IndexColorModel model = new IndexColorModel(bits, colors, rbmap, gmap, rbmap); // Create the pixels int pixels[] = new int[width * height]; int index = 0; for (int y = 0; y < height; y++) for (int x = 0; x < width; x++) pixels[index++] = (x * colors) / width; // Create the image img = createImage(new MemoryImageSource(width, height, model, pixels, 0, width)); } public void paint(Graphics g) { g.drawImage(img, 0, 0, this); } } It worked great but I tried to load a custom image jpeg mapped on my own colormap but it didnt work right. I saw only a bunch of green and blue pixels drawn on a white background. My custom color map method here: public void inintByteArrays() { double[][] c = // basic color map { { 0.0000, 0.0000, 0.5625 }, { 0.0000, 0.0000, 0.6250 }, { 0.0000, 0.0000, 0.6875 }, { 0.0000, 0.0000, 0.6875 }, { 0.0000, 0.0000, 0.7500 }, { 0.0000, 0.0000, 0.8125 }, { 0.0000, 0.0000, 0.8750 }, { 0.0000, 0.0000, 0.9375 }, { 0.0000, 0.0000, 1.0000 }, { 0.0000, 0.0625, 1.0000 }, { 0.0000, 0.1250, 1.0000 }, { 0.0000, 0.1875, 1.0000 }, { 0.0000, 0.2500, 1.0000 }, { 0.0000, 0.3125, 1.0000 }, { 0.0000, 0.3750, 1.0000 }, { 0.0000, 0.4375, 1.0000 }, { 0.0000, 0.5000, 1.0000 }, { 0.0000, 0.5625, 1.0000 }, { 0.0000, 0.6250, 1.0000 }, { 0.0000, 0.6875, 1.0000 }, { 0.0000, 0.7500, 1.0000 }, { 0.0000, 0.8125, 1.0000 }, { 0.0000, 0.8750, 1.0000 }, { 0.0000, 0.9375, 1.0000 }, { 0.0000, 1.0000, 1.0000 }, { 0.0625, 1.0000, 0.9375 }, { 0.1250, 1.0000, 0.8750 }, { 0.1875, 1.0000, 0.8125 }, { 0.2500, 1.0000, 0.7500 }, { 0.3125, 1.0000, 0.6875 }, { 0.3750, 1.0000, 0.6250 }, { 0.4375, 1.0000, 0.5625 }, { 0.5000, 1.0000, 0.5000 }, { 0.5625, 1.0000, 0.4375 }, { 0.6250, 1.0000, 0.3750 }, { 0.6875, 1.0000, 0.3125 }, { 0.7500, 1.0000, 0.2500 }, { 0.8125, 1.0000, 0.1875 }, { 0.8750, 1.0000, 0.1250 }, { 0.9375, 1.0000, 0.0625 }, { 1.0000, 1.0000, 0.0000 }, { 1.0000, 0.9375, 0.0000 }, { 1.0000, 0.8750, 0.0000 }, { 1.0000, 0.8125, 0.0000 }, { 1.0000, 0.7500, 0.0000 }, { 1.0000, 0.6875, 0.0000 }, { 1.0000, 0.6250, 0.0000 }, { 1.0000, 0.5625, 0.0000 }, { 1.0000, 0.5000, 0.0000 }, { 1.0000, 0.4375, 0.0000 }, { 1.0000, 0.3750, 0.0000 }, { 1.0000, 0.3125, 0.0000 }, { 1.0000, 0.2500, 0.0000 }, { 1.0000, 0.1875, 0.0000 }, { 1.0000, 0.1250, 0.0000 }, { 1.0000, 0.0625, 0.0000 }, { 1.0000, 0.0000, 0.0000 }, { 0.9375, 0.0000, 0.0000 }, { 0.8750, 0.0000, 0.0000 }, { 0.8125, 0.0000, 0.0000 }, { 0.7500, 0.0000, 0.0000 }, { 0.6875, 0.0000, 0.0000 }, { 0.6250, 0.0000, 0.0000 }, { 0.5625, 0.0000, 0.0000 }, { 0.5000, 0.0000, 0.0000 } }; for (int i = 0; i < c.length; i++) { for (int j = 0; j < c[i].length; j++) { if (j == 0) r[i] = (byte) ((byte) c[i][j]*255); if (j == 1) g[i] = (byte) ((byte) c[i][j]*255); if (j == 2) b[i] = (byte) ((byte) c[i][j]*255); } } My question is how I can use my colormap for any image I want to load and map in the right way. Thank you very much! Greetings, protein1.

    Read the article

  • php mysql cpanel high cpu usage

    - by Megahostzone Santu
    server taking high cpu usage load average: 108.87, 105.92, 85.82 netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n Reselt showing too much connect from server IP cpanel Process Manager showing 19.4 | 0.5 | /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --log-error=/var/lib/mysql/zebra546.serverstall.com.err --pid-file=/var/lib/mysql/zebra546.serverstall.com.pid 3.0 | 0.2 | /usr/bin/php /home/nowwatch/public_html/index.php

    Read the article

  • Looking for suggestions on how to reuse AlertDialogs to confirm actions of contextual menus on the items of an ActivityList

    - by Ozone
    I use a ListActivity to display a list of items. The user can long-press an item in the list to display a contextual menu. This menu contains among other things an option to delete the long-pressed item. When the user selects this option, a dialog pops up asking for confirmation of the deletion. Upon confirmation, the item is deleted from the list. I would like to reuse the AlertDialog as much as possible. My attempts at using onPrepareDialog(int, View, Bundle) have been defeated by the fact that the Bundle is not passed to the DialogInterface.OnClickListener. I end up having to recreate a listener on every invocation. I see several ways to solve this: recreate the dialog on every occasion (pros: simple, cons: wasteful) keep the DialogInterface.OnClickListener in a field on the ListActivity and keep the item to be deleted as a field of the listener. (pros: no memory waste, cons: need to manage state). Q: is this safe? have onPrepareDialog update the title, and bind new View.OnClickListeners on the buttons of the AlertDialog. (pros: limit waste, cons: new View.OnClickListener on every invocation). If DialogInterface.OnClickListener accepted a Bundle, I wouldn't have to jump through hoops to keep track of the item being deleted. This is not a blocker, but I would love to see an elegant solution. I would love to hear your suggestions :) Here is the code for option #1, if you want to play with this: public class Example extends ListActivity { private static final int CONFIRM_DELETE_DIALOG = 1; private static final String POSITION_KEY = "position"; private ArrayAdapter<String> mAdapter; @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); mAdapter = new ArrayAdapter<String>( this, android.R.layout.simple_list_item_1, new String[] { "one", "two" }); setListAdapter(mAdapter); registerForContextMenu(getListView()); } @Override public void onCreateContextMenu(ContextMenu menu, View v, ContextMenuInfo menuInfo) { super.onCreateContextMenu(menu, v, menuInfo); getMenuInflater().inflate(R.menu.my_lovely_menu, menu); } @Override public boolean onContextItemSelected(MenuItem item) { AdapterContextMenuInfo info = (AdapterContextMenuInfo) item.getMenuInfo(); switch (item.getItemId()) { case R.id.delete_item: Bundle bundle = new Bundle(); bundle.putInt(POSITION_KEY, info.position); showDialog(CONFIRM_DELETE_DIALOG, bundle); return true; default: return super.onContextItemSelected(item); } } @Override protected Dialog onCreateDialog(int id, Bundle args) { switch (id) { case CONFIRM_DELETE_DIALOG: final int position = args.getInt(POSITION_KEY); AlertDialog.Builder builder = new AlertDialog.Builder(); builder.setCancelable(false); builder.setTitle(String.format( getString(R.string.confirm_delete), mAdapter.getItem(position))); DialogInterface.OnClickListener listener = new DialogInterface.OnClickListener() { public void onClick(DialogInterface dialog, int which) { switch (which) { case DialogInterface.BUTTON_POSITIVE: mAdapter.remove(mAdapter.getItem(position)); // Dismiss the dialog to ensure OnDismissListeners are notified. dialog.dismiss(); break; case DialogInterface.BUTTON_NEGATIVE: // Cancel the dialog to ensure OnCancelListeners are notified. dialog.cancel(); break; } // Remove the dialog so it is re-created next time it is required. removeDialog(CONFIRM_DELETE_DIALOG); } }; builder.setPositiveButton(android.R.string.yes, listener); builder.setNegativeButton(android.R.string.no, listener); return builder.create(); default: return super.onCreateDialog(id, args); } } }

    Read the article

  • Is it possible to prevent windows in Mac OS from opening below the bottom edge of the screen?

    - by user31262
    When opening a new window in finder, the new window will go below the screen. This is not a big deal, but when using macvim or terminal it can be annoying since your command prompt is now hidden. Firefox seems to be smarter, it never overlaps below the screen. Same deal for the right hand side of the screen, windows often open with their right edge cut off. Is there any way to prevent this?

    Read the article

  • Useful keyboard shortcuts on a Mac

    - by warren
    Most Mac OS users know about ⌘-Q for quit. And ⌘-C, ⌘-X, ⌘-V for copy cut and paste. Likewsie ⌘-P for print and ⌘-S for save. Is there a keyboard shortcut for maximizing a window? Or minimizing it? Are there other daily-use keyboard shortcuts you use on your Mac?

    Read the article

  • How to jump back to the first character in *nix command line?

    - by clami219
    When writing a long command in the *nix command line and having to go back to the first character, in order to add something at the beginning (for instance a nohup, when you realize the process will be a long one, or a sudo, when you realize you need root permissions) it can take a long time for the cursor to make its way back to the first character... Is there a short cut that allows you to jump straight there? I'm using a mac, so Home is not an option

    Read the article

  • Java: Object Array assignment in for loop

    - by Hackster
    I am trying to use Dijkstra's algorithm to find the shortest path from a specific vertex (v0) to the rest of them. That is solved and works well with this code from this link below: http://en.literateprograms.org/index.php?title=Special:DownloadCode/Dijkstra%27s_algorithm_(Java)&oldid=15444 I am having trouble with assigning the Edge array in a for loop from the user input, as opposed to hard-coding it like it is here. Any help assigning a new edge to Edge[] adjacencies from each vertex? Keeping in mind it could be 1 or multiple edges. class Vertex implements Comparable<Vertex> { public final String name; public Edge[] adjacencies; public double minDistance = Double.POSITIVE_INFINITY; public Vertex previous; public Vertex(String argName) { name = argName; } public String toString() { return name; } public int compareTo(Vertex other){ return Double.compare(minDistance, other.minDistance); } } class Edge{ public final Vertex target; public final double weight; public Edge(Vertex argTarget, double argWeight){ target = argTarget; weight = argWeight; } } public static void main(String[] args) { Vertex v[] = new Vertex[3]; Vertex v[0] = new Vertex("Harrisburg"); Vertex v[1] = new Vertex("Baltimore"); Vertex v[2] = new Vertex("Washington"); v0.adjacencies = new Edge[]{ new Edge(v[1], 1), new Edge(v[2], 3) }; v1.adjacencies = new Edge[]{ new Edge(v[0], 1), new Edge(v[2], 1),}; v2.adjacencies = new Edge[]{ new Edge(v[0], 3), new Edge(v[1], 1) }; Vertex[] vertices = { v0, v1, v2}; /*Three vertices with weight: V0 connects (V1,1),(V2,3) V1 connects (V0,1),(V2,1) V2 connects (V1,1),(V2,3) */ computePaths(v0); for (Vertex v : vertices){ System.out.println("Distance to " + v + ": " + v.minDistance); List<Vertex> path = getShortestPathTo(v); System.out.println("Path: " + path); } } } The above code works well in finding the shortest path from v0 to all the other vertices. The problem occurs when assigning the new edge[] to edge[] adjacencies. For example this does not produce the correct output: for (int i = 0; i < total_vertices; i++){ s = br.readLine(); char[] line = s.toCharArray(); for (int j = 0; j < line.length; j++){ if(j % 4 == 0 ){ //Input: vertex weight vertex weight: 1 1 2 3 int vert = Integer.parseInt(String.valueOf(line[j])); int w = Integer.parseInt(String.valueOf(line[j+2])); v[i].adjacencies = new Edge[] {new Edge(v[vert], w)}; } } } As opposed to this: v0.adjacencies = new Edge[]{ new Edge(v[1], 1), new Edge(v[2], 3) }; How can I take the user input and make an Edge[], to pass it to adjacencies? The problem is it could be 0 edges or many. Any help would be much appreciated Thanks!

    Read the article

  • Parallelism in .NET – Part 4, Imperative Data Parallelism: Aggregation

    - by Reed
    In the article on simple data parallelism, I described how to perform an operation on an entire collection of elements in parallel.  Often, this is not adequate, as the parallel operation is going to be performing some form of aggregation. Simple examples of this might include taking the sum of the results of processing a function on each element in the collection, or finding the minimum of the collection given some criteria.  This can be done using the techniques described in simple data parallelism, however, special care needs to be taken into account to synchronize the shared data appropriately.  The Task Parallel Library has tools to assist in this synchronization. The main issue with aggregation when parallelizing a routine is that you need to handle synchronization of data.  Since multiple threads will need to write to a shared portion of data.  Suppose, for example, that we wanted to parallelize a simple loop that looked for the minimum value within a dataset: double min = double.MaxValue; foreach(var item in collection) { double value = item.PerformComputation(); min = System.Math.Min(min, value); } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } This seems like a good candidate for parallelization, but there is a problem here.  If we just wrap this into a call to Parallel.ForEach, we’ll introduce a critical race condition, and get the wrong answer.  Let’s look at what happens here: // Buggy code! Do not use! double min = double.MaxValue; Parallel.ForEach(collection, item => { double value = item.PerformComputation(); min = System.Math.Min(min, value); }); This code has a fatal flaw: min will be checked, then set, by multiple threads simultaneously.  Two threads may perform the check at the same time, and set the wrong value for min.  Say we get a value of 1 in thread 1, and a value of 2 in thread 2, and these two elements are the first two to run.  If both hit the min check line at the same time, both will determine that min should change, to 1 and 2 respectively.  If element 1 happens to set the variable first, then element 2 sets the min variable, we’ll detect a min value of 2 instead of 1.  This can lead to wrong answers. Unfortunately, fixing this, with the Parallel.ForEach call we’re using, would require adding locking.  We would need to rewrite this like: // Safe, but slow double min = double.MaxValue; // Make a "lock" object object syncObject = new object(); Parallel.ForEach(collection, item => { double value = item.PerformComputation(); lock(syncObject) min = System.Math.Min(min, value); }); This will potentially add a huge amount of overhead to our calculation.  Since we can potentially block while waiting on the lock for every single iteration, we will most likely slow this down to where it is actually quite a bit slower than our serial implementation.  The problem is the lock statement – any time you use lock(object), you’re almost assuring reduced performance in a parallel situation.  This leads to two observations I’ll make: When parallelizing a routine, try to avoid locks. That being said: Always add any and all required synchronization to avoid race conditions. These two observations tend to be opposing forces – we often need to synchronize our algorithms, but we also want to avoid the synchronization when possible.  Looking at our routine, there is no way to directly avoid this lock, since each element is potentially being run on a separate thread, and this lock is necessary in order for our routine to function correctly every time. However, this isn’t the only way to design this routine to implement this algorithm.  Realize that, although our collection may have thousands or even millions of elements, we have a limited number of Processing Elements (PE).  Processing Element is the standard term for a hardware element which can process and execute instructions.  This typically is a core in your processor, but many modern systems have multiple hardware execution threads per core.  The Task Parallel Library will not execute the work for each item in the collection as a separate work item. Instead, when Parallel.ForEach executes, it will partition the collection into larger “chunks” which get processed on different threads via the ThreadPool.  This helps reduce the threading overhead, and help the overall speed.  In general, the Parallel class will only use one thread per PE in the system. Given the fact that there are typically fewer threads than work items, we can rethink our algorithm design.  We can parallelize our algorithm more effectively by approaching it differently.  Because the basic aggregation we are doing here (Min) is communitive, we do not need to perform this in a given order.  We knew this to be true already – otherwise, we wouldn’t have been able to parallelize this routine in the first place.  With this in mind, we can treat each thread’s work independently, allowing each thread to serially process many elements with no locking, then, after all the threads are complete, “merge” together the results. This can be accomplished via a different set of overloads in the Parallel class: Parallel.ForEach<TSource,TLocal>.  The idea behind these overloads is to allow each thread to begin by initializing some local state (TLocal).  The thread will then process an entire set of items in the source collection, providing that state to the delegate which processes an individual item.  Finally, at the end, a separate delegate is run which allows you to handle merging that local state into your final results. To rewriting our routine using Parallel.ForEach<TSource,TLocal>, we need to provide three delegates instead of one.  The most basic version of this function is declared as: public static ParallelLoopResult ForEach<TSource, TLocal>( IEnumerable<TSource> source, Func<TLocal> localInit, Func<TSource, ParallelLoopState, TLocal, TLocal> body, Action<TLocal> localFinally ) The first delegate (the localInit argument) is defined as Func<TLocal>.  This delegate initializes our local state.  It should return some object we can use to track the results of a single thread’s operations. The second delegate (the body argument) is where our main processing occurs, although now, instead of being an Action<T>, we actually provide a Func<TSource, ParallelLoopState, TLocal, TLocal> delegate.  This delegate will receive three arguments: our original element from the collection (TSource), a ParallelLoopState which we can use for early termination, and the instance of our local state we created (TLocal).  It should do whatever processing you wish to occur per element, then return the value of the local state after processing is completed. The third delegate (the localFinally argument) is defined as Action<TLocal>.  This delegate is passed our local state after it’s been processed by all of the elements this thread will handle.  This is where you can merge your final results together.  This may require synchronization, but now, instead of synchronizing once per element (potentially millions of times), you’ll only have to synchronize once per thread, which is an ideal situation. Now that I’ve explained how this works, lets look at the code: // Safe, and fast! double min = double.MaxValue; // Make a "lock" object object syncObject = new object(); Parallel.ForEach( collection, // First, we provide a local state initialization delegate. () => double.MaxValue, // Next, we supply the body, which takes the original item, loop state, // and local state, and returns a new local state (item, loopState, localState) => { double value = item.PerformComputation(); return System.Math.Min(localState, value); }, // Finally, we provide an Action<TLocal>, to "merge" results together localState => { // This requires locking, but it's only once per used thread lock(syncObj) min = System.Math.Min(min, localState); } ); Although this is a bit more complicated than the previous version, it is now both thread-safe, and has minimal locking.  This same approach can be used by Parallel.For, although now, it’s Parallel.For<TLocal>.  When working with Parallel.For<TLocal>, you use the same triplet of delegates, with the same purpose and results. Also, many times, you can completely avoid locking by using a method of the Interlocked class to perform the final aggregation in an atomic operation.  The MSDN example demonstrating this same technique using Parallel.For uses the Interlocked class instead of a lock, since they are doing a sum operation on a long variable, which is possible via Interlocked.Add. By taking advantage of local state, we can use the Parallel class methods to parallelize algorithms such as aggregation, which, at first, may seem like poor candidates for parallelization.  Doing so requires careful consideration, and often requires a slight redesign of the algorithm, but the performance gains can be significant if handled in a way to avoid excessive synchronization.

    Read the article

  • Java Logger API

    - by Koppar
    This is a more like a tip rather than technical write up and serves as a quick intro for newbies. The logger API helps to diagnose application level or JDK level issues at runtime. There are 7 levels which decide the detailing in logging (SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST). Its best to start with highest level and as we narrow down, use more detailed logging for a specific area. SEVERE is the highest and FINEST is the lowest. This may not make sense until we understand some jargon. The Logger class provides the ability to stream messages to an output stream in a format that can be controlled by the user. What this translates to is, I can create a logger with this simple invocation and use it add debug messages in my class: import java.util.logging.*; private static final Logger focusLog = Logger.getLogger("java.awt.focus.KeyboardFocusManager"); if (focusLog.isLoggable(Level.FINEST)) { focusLog.log(Level.FINEST, "Calling peer setCurrentFocusOwner}); LogManager acts like a book keeper and all the getLogger calls are forwarded to LogManager. The LogManager itself is a singleton class object which gets statically initialized on JVM start up. More on this later. If there is no existing logger with the given name, a new one is created. If there is one (and not yet GC’ed), then the existing Logger object is returned. By default, a root logger is created on JVM start up. All anonymous loggers are made as the children of the root logger. Named loggers have the hierarchy as per their name resolutions. Eg: java.awt.focus is the parent logger for java.awt.focus.KeyboardFocusManager etc. Before logging any message, the logger checks for the log level specified. If null is specified, the log level of the parent logger will be set. However, if the log level is off, no log messages would be written, irrespective of the parent’s log level. All the messages that are posted to the Logger are handled as a LogRecord object.i.e. FocusLog.log would create a new LogRecord object with the log level and message as its data members). The level of logging and thread number are also tracked. LogRecord is passed on to all the registered Handlers. Handler is basically a means to output the messages. The output may be redirected to either a log file or console or a network logging service. The Handler classes use the LogManager properties to set filters and formatters. During initialization or JVM start up, LogManager looks for logging.properties file in jre/lib and sets the properties if the file is provided. An alternate location for properties file can also be specified by setting java.util.logging.config.file system property. This can be set in Java Control Panel ? Java ? Runtime parameters as -Djava.util.logging.config.file = <mylogfile> or passed as a command line parameter java -Djava.util.logging.config.file = C:/Sunita/myLog The redirection of logging depends on what is specified rather registered as a handler with JVM in the properties file. java.util.logging.ConsoleHandler sends the output to system.err and java.util.logging.FileHandler sends the output to file. File name of the log file can also be specified. If you prefer XML format output, in the configuration file, set java.util.logging.FileHandler.formatter = java.util.logging.XMLFormatter and if you prefer simple text, set set java.util.logging.FileHandler.formatter =java.util.logging.SimpleFormatter Below is the default logging Configuration file: ############################################################ # Default Logging Configuration File # You can use a different file by specifying a filename # with the java.util.logging.config.file system property. # For example java -Djava.util.logging.config.file=myfile ############################################################ ############################################################ # Global properties ############################################################ # "handlers" specifies a comma separated list of log Handler # classes. These handlers will be installed during VM startup. # Note that these classes must be on the system classpath. # By default we only configure a ConsoleHandler, which will only # show messages at the INFO and above levels. handlers= java.util.logging.ConsoleHandler # To also add the FileHandler, use the following line instead. #handlers= java.util.logging.FileHandler, java.util.logging.ConsoleHandler # Default global logging level. # This specifies which kinds of events are logged across # all loggers. For any given facility this global level # can be overriden by a facility specific level # Note that the ConsoleHandler also has a separate level # setting to limit messages printed to the console. .level= INFO ############################################################ # Handler specific properties. # Describes specific configuration info for Handlers. ############################################################ # default file output is in user's home directory. java.util.logging.FileHandler.pattern = %h/java%u.log java.util.logging.FileHandler.limit = 50000 java.util.logging.FileHandler.count = 1 java.util.logging.FileHandler.formatter = java.util.logging.XMLFormatter # Limit the message that are printed on the console to INFO and above. java.util.logging.ConsoleHandler.level = INFO java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter ############################################################ # Facility specific properties. # Provides extra control for each logger. ############################################################ # For example, set the com.xyz.foo logger to only log SEVERE # messages: com.xyz.foo.level = SEVERE Since I primarily use this method to track focus issues, here is how I get detailed awt focus related logging. Just set the logger name to java.awt.focus.level=FINEST and change the default log level to FINEST. Below is a basic sample program. The sample programs are from http://www2.cs.uic.edu/~sloan/CLASSES/java/ and have been modified to illustrate the logging API. By changing the .level property in the logging.properties file, one can control the output written to the logs. To play around with the example, try changing the levels in the logging.properties file and notice the difference in messages going to the log file. Example --------KeyboardReader.java------------------------------------------------------------------------------------- import java.io.*; import java.util.*; import java.util.logging.*; public class KeyboardReader { private static final Logger mylog = Logger.getLogger("samples.input"); public static void main (String[] args) throws java.io.IOException { String s1; String s2; double num1, num2, product; // set up the buffered reader to read from the keyboard BufferedReader br = new BufferedReader (new InputStreamReader (System.in)); System.out.println ("Enter a line of input"); s1 = br.readLine(); if (mylog.isLoggable(Level.SEVERE)) { mylog.log (Level.SEVERE,"The line entered is " + s1); } if (mylog.isLoggable(Level.INFO)) { mylog.log (Level.INFO,"The line has " + s1.length() + " characters"); } if (mylog.isLoggable(Level.FINE)) { mylog.log (Level.FINE,"Breaking the line into tokens we get:"); } int numTokens = 0; StringTokenizer st = new StringTokenizer (s1); while (st.hasMoreTokens()) { s2 = st.nextToken(); numTokens++; if (mylog.isLoggable(Level.FINEST)) { mylog.log (Level.FINEST, " Token " + numTokens + " is: " + s2); } } } } ----------MyFileReader.java---------------------------------------------------------------------------------------- import java.io.*; import java.util.*; import java.util.logging.*; public class MyFileReader extends KeyboardReader { private static final Logger mylog = Logger.getLogger("samples.input.file"); public static void main (String[] args) throws java.io.IOException { String s1; String s2; // set up the buffered reader to read from the keyboard BufferedReader br = new BufferedReader (new FileReader ("MyFileReader.txt")); s1 = br.readLine(); if (mylog.isLoggable(Level.SEVERE)) { mylog.log (Level.SEVERE,"ATTN The line is " + s1); } if (mylog.isLoggable(Level.INFO)) { mylog.log (Level.INFO, "The line has " + s1.length() + " characters"); } if (mylog.isLoggable(Level.FINE)) { mylog.log (Level.FINE,"Breaking the line into tokens we get:"); } int numTokens = 0; StringTokenizer st = new StringTokenizer (s1); while (st.hasMoreTokens()) { s2 = st.nextToken(); numTokens++; if (mylog.isLoggable(Level.FINEST)) { mylog.log (Level.FINEST,"Breaking the line into tokens we get:"); mylog.log (Level.FINEST," Token " + numTokens + " is: " + s2); } } //end of while } // end of main } // end of class ----------MyFileReader.txt------------------------------------------------------------------------------------------ My first logging example -------logging.properties------------------------------------------------------------------------------------------- handlers= java.util.logging.ConsoleHandler, java.util.logging.FileHandler .level= FINEST java.util.logging.FileHandler.pattern = java%u.log java.util.logging.FileHandler.limit = 50000 java.util.logging.FileHandler.count = 1 java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter java.util.logging.ConsoleHandler.level = FINEST java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter java.awt.focus.level=ALL ------Output log------------------------------------------------------------------------------------------- May 21, 2012 11:44:55 AM MyFileReader main SEVERE: ATTN The line is My first logging example May 21, 2012 11:44:55 AM MyFileReader main INFO: The line has 24 characters May 21, 2012 11:44:55 AM MyFileReader main FINE: Breaking the line into tokens we get: May 21, 2012 11:44:55 AM MyFileReader main FINEST: Breaking the line into tokens we get: May 21, 2012 11:44:55 AM MyFileReader main FINEST: Token 1 is: My May 21, 2012 11:44:55 AM MyFileReader main FINEST: Breaking the line into tokens we get: May 21, 2012 11:44:55 AM MyFileReader main FINEST: Token 2 is: first May 21, 2012 11:44:55 AM MyFileReader main FINEST: Breaking the line into tokens we get: May 21, 2012 11:44:55 AM MyFileReader main FINEST: Token 3 is: logging May 21, 2012 11:44:55 AM MyFileReader main FINEST: Breaking the line into tokens we get: May 21, 2012 11:44:55 AM MyFileReader main FINEST: Token 4 is: example Invocation command: "C:\Program Files (x86)\Java\jdk1.6.0_29\bin\java.exe" -Djava.util.logging.config.file=logging.properties MyFileReader References Further technical details are available here: http://docs.oracle.com/javase/1.4.2/docs/guide/util/logging/overview.html#1.0 http://docs.oracle.com/javase/1.4.2/docs/api/java/util/logging/package-summary.html http://www2.cs.uic.edu/~sloan/CLASSES/java/

    Read the article

  • Heaps of Trouble?

    - by Paul White NZ
    If you’re not already a regular reader of Brad Schulz’s blog, you’re missing out on some great material.  In his latest entry, he is tasked with optimizing a query run against tables that have no indexes at all.  The problem is, predictably, that performance is not very good.  The catch is that we are not allowed to create any indexes (or even new statistics) as part of our optimization efforts. In this post, I’m going to look at the problem from a slightly different angle, and present an alternative solution to the one Brad found.  Inevitably, there’s going to be some overlap between our entries, and while you don’t necessarily need to read Brad’s post before this one, I do strongly recommend that you read it at some stage; he covers some important points that I won’t cover again here. The Example We’ll use data from the AdventureWorks database, copied to temporary unindexed tables.  A script to create these structures is shown below: CREATE TABLE #Custs ( CustomerID INTEGER NOT NULL, TerritoryID INTEGER NULL, CustomerType NCHAR(1) COLLATE SQL_Latin1_General_CP1_CI_AI NOT NULL, ); GO CREATE TABLE #Prods ( ProductMainID INTEGER NOT NULL, ProductSubID INTEGER NOT NULL, ProductSubSubID INTEGER NOT NULL, Name NVARCHAR(50) COLLATE SQL_Latin1_General_CP1_CI_AI NOT NULL, ); GO CREATE TABLE #OrdHeader ( SalesOrderID INTEGER NOT NULL, OrderDate DATETIME NOT NULL, SalesOrderNumber NVARCHAR(25) COLLATE SQL_Latin1_General_CP1_CI_AI NOT NULL, CustomerID INTEGER NOT NULL, ); GO CREATE TABLE #OrdDetail ( SalesOrderID INTEGER NOT NULL, OrderQty SMALLINT NOT NULL, LineTotal NUMERIC(38,6) NOT NULL, ProductMainID INTEGER NOT NULL, ProductSubID INTEGER NOT NULL, ProductSubSubID INTEGER NOT NULL, ); GO INSERT #Custs ( CustomerID, TerritoryID, CustomerType ) SELECT C.CustomerID, C.TerritoryID, C.CustomerType FROM AdventureWorks.Sales.Customer C WITH (TABLOCK); GO INSERT #Prods ( ProductMainID, ProductSubID, ProductSubSubID, Name ) SELECT P.ProductID, P.ProductID, P.ProductID, P.Name FROM AdventureWorks.Production.Product P WITH (TABLOCK); GO INSERT #OrdHeader ( SalesOrderID, OrderDate, SalesOrderNumber, CustomerID ) SELECT H.SalesOrderID, H.OrderDate, H.SalesOrderNumber, H.CustomerID FROM AdventureWorks.Sales.SalesOrderHeader H WITH (TABLOCK); GO INSERT #OrdDetail ( SalesOrderID, OrderQty, LineTotal, ProductMainID, ProductSubID, ProductSubSubID ) SELECT D.SalesOrderID, D.OrderQty, D.LineTotal, D.ProductID, D.ProductID, D.ProductID FROM AdventureWorks.Sales.SalesOrderDetail D WITH (TABLOCK); The query itself is a simple join of the four tables: SELECT P.ProductMainID AS PID, P.Name, D.OrderQty, H.SalesOrderNumber, H.OrderDate, C.TerritoryID FROM #Prods P JOIN #OrdDetail D ON P.ProductMainID = D.ProductMainID AND P.ProductSubID = D.ProductSubID AND P.ProductSubSubID = D.ProductSubSubID JOIN #OrdHeader H ON D.SalesOrderID = H.SalesOrderID JOIN #Custs C ON H.CustomerID = C.CustomerID ORDER BY P.ProductMainID ASC OPTION (RECOMPILE, MAXDOP 1); Remember that these tables have no indexes at all, and only the single-column sampled statistics SQL Server automatically creates (assuming default settings).  The estimated query plan produced for the test query looks like this (click to enlarge): The Problem The problem here is one of cardinality estimation – the number of rows SQL Server expects to find at each step of the plan.  The lack of indexes and useful statistical information means that SQL Server does not have the information it needs to make a good estimate.  Every join in the plan shown above estimates that it will produce just a single row as output.  Brad covers the factors that lead to the low estimates in his post. In reality, the join between the #Prods and #OrdDetail tables will produce 121,317 rows.  It should not surprise you that this has rather dire consequences for the remainder of the query plan.  In particular, it makes a nonsense of the optimizer’s decision to use Nested Loops to join to the two remaining tables.  Instead of scanning the #OrdHeader and #Custs tables once (as it expected), it has to perform 121,317 full scans of each.  The query takes somewhere in the region of twenty minutes to run to completion on my development machine. A Solution At this point, you may be thinking the same thing I was: if we really are stuck with no indexes, the best we can do is to use hash joins everywhere. We can force the exclusive use of hash joins in several ways, the two most common being join and query hints.  A join hint means writing the query using the INNER HASH JOIN syntax; using a query hint involves adding OPTION (HASH JOIN) at the bottom of the query.  The difference is that using join hints also forces the order of the join, whereas the query hint gives the optimizer freedom to reorder the joins at its discretion. Adding the OPTION (HASH JOIN) hint results in this estimated plan: That produces the correct output in around seven seconds, which is quite an improvement!  As a purely practical matter, and given the rigid rules of the environment we find ourselves in, we might leave things there.  (We can improve the hashing solution a bit – I’ll come back to that later on). Faster Nested Loops It might surprise you to hear that we can beat the performance of the hash join solution shown above using nested loops joins exclusively, and without breaking the rules we have been set. The key to this part is to realize that a condition like (A = B) can be expressed as (A <= B) AND (A >= B).  Armed with this tremendous new insight, we can rewrite the join predicates like so: SELECT P.ProductMainID AS PID, P.Name, D.OrderQty, H.SalesOrderNumber, H.OrderDate, C.TerritoryID FROM #OrdDetail D JOIN #OrdHeader H ON D.SalesOrderID >= H.SalesOrderID AND D.SalesOrderID <= H.SalesOrderID JOIN #Custs C ON H.CustomerID >= C.CustomerID AND H.CustomerID <= C.CustomerID JOIN #Prods P ON P.ProductMainID >= D.ProductMainID AND P.ProductMainID <= D.ProductMainID AND P.ProductSubID = D.ProductSubID AND P.ProductSubSubID = D.ProductSubSubID ORDER BY D.ProductMainID OPTION (RECOMPILE, LOOP JOIN, MAXDOP 1, FORCE ORDER); I’ve also added LOOP JOIN and FORCE ORDER query hints to ensure that only nested loops joins are used, and that the tables are joined in the order they appear.  The new estimated execution plan is: This new query runs in under 2 seconds. Why Is It Faster? The main reason for the improvement is the appearance of the eager Index Spools, which are also known as index-on-the-fly spools.  If you read my Inside The Optimiser series you might be interested to know that the rule responsible is called JoinToIndexOnTheFly. An eager index spool consumes all rows from the table it sits above, and builds a index suitable for the join to seek on.  Taking the index spool above the #Custs table as an example, it reads all the CustomerID and TerritoryID values with a single scan of the table, and builds an index keyed on CustomerID.  The term ‘eager’ means that the spool consumes all of its input rows when it starts up.  The index is built in a work table in tempdb, has no associated statistics, and only exists until the query finishes executing. The result is that each unindexed table is only scanned once, and just for the columns necessary to build the temporary index.  From that point on, every execution of the inner side of the join is answered by a seek on the temporary index – not the base table. A second optimization is that the sort on ProductMainID (required by the ORDER BY clause) is performed early, on just the rows coming from the #OrdDetail table.  The optimizer has a good estimate for the number of rows it needs to sort at that stage – it is just the cardinality of the table itself.  The accuracy of the estimate there is important because it helps determine the memory grant given to the sort operation.  Nested loops join preserves the order of rows on its outer input, so sorting early is safe.  (Hash joins do not preserve order in this way, of course). The extra lazy spool on the #Prods branch is a further optimization that avoids executing the seek on the temporary index if the value being joined (the ‘outer reference’) hasn’t changed from the last row received on the outer input.  It takes advantage of the fact that rows are still sorted on ProductMainID, so if duplicates exist, they will arrive at the join operator one after the other. The optimizer is quite conservative about introducing index spools into a plan, because creating and dropping a temporary index is a relatively expensive operation.  It’s presence in a plan is often an indication that a useful index is missing. I want to stress that I rewrote the query in this way primarily as an educational exercise – I can’t imagine having to do something so horrible to a production system. Improving the Hash Join I promised I would return to the solution that uses hash joins.  You might be puzzled that SQL Server can create three new indexes (and perform all those nested loops iterations) faster than it can perform three hash joins.  The answer, again, is down to the poor information available to the optimizer.  Let’s look at the hash join plan again: Two of the hash joins have single-row estimates on their build inputs.  SQL Server fixes the amount of memory available for the hash table based on this cardinality estimate, so at run time the hash join very quickly runs out of memory. This results in the join spilling hash buckets to disk, and any rows from the probe input that hash to the spilled buckets also get written to disk.  The join process then continues, and may again run out of memory.  This is a recursive process, which may eventually result in SQL Server resorting to a bailout join algorithm, which is guaranteed to complete eventually, but may be very slow.  The data sizes in the example tables are not large enough to force a hash bailout, but it does result in multiple levels of hash recursion.  You can see this for yourself by tracing the Hash Warning event using the Profiler tool. The final sort in the plan also suffers from a similar problem: it receives very little memory and has to perform multiple sort passes, saving intermediate runs to disk (the Sort Warnings Profiler event can be used to confirm this).  Notice also that because hash joins don’t preserve sort order, the sort cannot be pushed down the plan toward the #OrdDetail table, as in the nested loops plan. Ok, so now we understand the problems, what can we do to fix it?  We can address the hash spilling by forcing a different order for the joins: SELECT P.ProductMainID AS PID, P.Name, D.OrderQty, H.SalesOrderNumber, H.OrderDate, C.TerritoryID FROM #Prods P JOIN #Custs C JOIN #OrdHeader H ON H.CustomerID = C.CustomerID JOIN #OrdDetail D ON D.SalesOrderID = H.SalesOrderID ON P.ProductMainID = D.ProductMainID AND P.ProductSubID = D.ProductSubID AND P.ProductSubSubID = D.ProductSubSubID ORDER BY D.ProductMainID OPTION (MAXDOP 1, HASH JOIN, FORCE ORDER); With this plan, each of the inputs to the hash joins has a good estimate, and no hash recursion occurs.  The final sort still suffers from the one-row estimate problem, and we get a single-pass sort warning as it writes rows to disk.  Even so, the query runs to completion in three or four seconds.  That’s around half the time of the previous hashing solution, but still not as fast as the nested loops trickery. Final Thoughts SQL Server’s optimizer makes cost-based decisions, so it is vital to provide it with accurate information.  We can’t really blame the performance problems highlighted here on anything other than the decision to use completely unindexed tables, and not to allow the creation of additional statistics. I should probably stress that the nested loops solution shown above is not one I would normally contemplate in the real world.  It’s there primarily for its educational and entertainment value.  I might perhaps use it to demonstrate to the sceptical that SQL Server itself is crying out for an index. Be sure to read Brad’s original post for more details.  My grateful thanks to him for granting permission to reuse some of his material. Paul White Email: [email protected] Twitter: @PaulWhiteNZ

    Read the article

  • CodePlex Daily Summary for Saturday, June 05, 2010

    CodePlex Daily Summary for Saturday, June 05, 2010New Projects555 Calculator: A simple calculator to help choosing resistor and capacitor values to get the frequency you're looking for on a 555 timer.BleQua .NET: PL: Program sieciowy BleQua .NET jest multi-komunikatorem. EN: Network program BleQua .NET is multicomunicator.ChatDiplomaWork: ChatSample is the sample project.CSUFVGDC Summer Jam: Repository for VGDC summer game jam.Database Export Wizard: ExportWizard is a Step Wizard for Database Export using ASP.net and SQL Server. It allows easy export in any of the standard formats: CSV, TXT, HTM...Dozer Enterprise Library for .NET: a light .net framework for enterprise applications developmentEmployee Management System: This is an Employee Management System. the goal here is to offer a software that caters to small to mid sized businesses for free. This program a...Fanray: My project on Codeplex.Infragistics Analytics Framework: This project includes wrappers for the Infragistics controls that integrate with the recently launched Microsoft Silverlight Analytics Framework. ...KIME Simio Extensions: This is the official project of KIME Solutions. Here you find any current developments of KIME that are publicly available. KIME develops extension...MindTouch Community Extensions: MindTouch Community Extentions is a Native C# extention library for MindTouch Core that will have Dekiscript functions for full coverage of the API...NginxTray: NginxTray allows you manage easily Nginx Web Server by a Tray icon.NStore: NStore is a virtual store example done with ASP.NET MVC 2.0 tecnologyProjet Campus Numerique + Appli Mobile: Projet de création d'un campus numérique pour l'ISEN et d'un application mobile d'accès.SharePoint 2010 Feature Upgrade Kit: A set of tools for managing upgradable Features in SharePoint 2010. (Upgrading Features is a means of deploying code/artifact updates to existing S...SiteMap Utility for DNN Blog Module: This is a mini-project which allows you to easily add or generate an XML site map to your DotNetNuke® website for the search engines to use to inde...Space Explorer: A small app to help users examine folders to see which files and subfolders are taking up space. Still in development, no releases available curren...SQL Compact Toolbox: SQL Compact Toolbox is a Visual Studio 2010 add-in, that adds scripting, import, export, migrate, rename, run script and other upcoming SQL Server ...Twilverlight: Twliverlight is TweetDeck mixed with Silverlight. Much as I like using TweetDeck, it hogs my memory out, so this is an attempt to write a memory-ef...Visual Studio 2010 FxCop Extension: Visual Studio 2010 FxCop Extension allows to integrate stand-alone FxCop into Visual Studio 2010. You'll be able to analysis your source code with ...VisualStudio 2010 JavaScript Outlining: Visual Studio 2010 editor extension for JavaScript code blocks and custom regions outlining Wiki Shelf: Wiki Shelf is a Wikipedia browser app. The goal is to bring the library experience of browsing books, studying, and researching to the Wikipedia u...X-Arena - Magic: Projeto de PDS2 no Curso de Tecnologia em Desenvolvimento de Software no CEFETRN. O desenvolvimento do jogo Quiz Arena possui como pr...New Releases555 Calculator: 555Calc release v1.0: The initial 1 point uh-oh release of 555Calc.BleQua .NET: BleQua .NET 1.0.0.0: First releaseChatterbot JBot: JBot 1.0.1.155: Change presentation technology from Window Forms to Widndown Presentation Fundation.Community Forums NNTP bridge: Community Forums NNTP Bridge V26: Release of the Community Forums NNTP Bridge to access the social and anwsers MS forums with a single, open source NNTP bridge. This release has ad...Community Forums NNTP bridge: Community Forums NNTP Bridge V27: Release of the Community Forums NNTP Bridge to access the social and anwsers MS forums with a single, open source NNTP bridge. This release has ad...Community Forums NNTP bridge: Community Forums NNTP Bridge V28: Release of the Community Forums NNTP Bridge to access the social and anwsers MS forums with a single, open source NNTP bridge. This release has ad...CSS 360 Planetary Calendar: Final Release: =============================================================================== Final Release Version: 2.0 Tools Used: - Collaboration, Releas...Database Export Wizard: Version 3: As described in CodeProject article: Database Export Wizard for ASP.net and SQL Server. http://www.codeproject.com/KB/aspnet/DatabaseExportWizard.aspxEdu Math: Edu Math 2.0.10.122: Change version .NET Framework.Employee Management System: V1 (beta): This version is still in beta testing. Any issues, comments or suggestions are greatly appreciated. The export to excel function in this release o...ESB.NET: ESBDeploy_7.0.27.0 (x64 and x86) [ESB.NET 7.0 RC1]: Release Details Changes Since Last Release (since 6.3.47.1) - Targets .NET Framework 4, Visual Studio.NET 2010, Workflow 4 - Flowchart workflow ada...Free Silverlight & WPF Chart Control - Visifire: Visifire SL and WPF Charts v3.1.1 beta 2 Released: Hi, This release contains the following enhancements: *ShowIndicator() and HideIndicator() function has been implemented in Chart. So now user wi...Free Silverlight & WPF Chart Control - Visifire: Visifire SL and WPF Charts v3.5.4 beta 2 Released: Hi, This release contains the following enhancements: *ShowIndicator() and HideIndicator() function has been implemented in Chart. So now user wi...FsCheck: A random testing framework: FsCheck 0.7: What to download? If you use F# April 2010 CTP with Visual Studio 2008 or Visual Studio 2010, either Source or Binaries will do. To open source in...Git Source Control Provider: V 0.5: For VS 2010 users, it is recommanded to install it within Visual Studio by selecting Tools | Extension Manager. Run Visual Studio. Go to Tools ...GPdotNET - Genetic Programming Tool: GPdotNETv0.95: 1. Localization support 2. Export functionality for GP Model with training and testing data. 3. Export GPModel with Testing and Training data. 4....JoshDOS: JoshDOS 1.1: 1.1 adds a toutorial of how to create new commands and of course, you need the COMSOS user kit or dev kit. Ver 1.1 also includes a demo called Gue...KIME Simio Extensions: KIME.SimioDebugStep: This simple Simio step allows you to debug any number of expressions from within the simulation run. The debug information is displayed using a mes...NginxTray: NginxTray 0.3 Beta 2: NginxTray 0.3 Beta 2NginxTray: NginxTray 0.5 Beta 3: NginxTray 0.5 Beta 3NginxTray: NginxTray 0.6 RC1: NginxTray 0.6 RC1Open Source PLM Activities: Prodeos_OC beta 1.0: The “Innovator – MS Office Connector” is a product developed by Prodeos (www.prodeos.com). It is a light connector made to facilitate the use of Mi...Paint.NET PSD Plugin: 1.5.1: Changes in this release: Bitmap-mode images can now be loaded. Thanks to dhnc for filing the bug. Plugin no longer crashes on files with user m...SharePoint 2010 Feature Upgrade Kit: 1.0.0.0: This release contains:- - Custom application page to manage the upgrade of Site and Web-scoped Features. To come in the next release: - Companio...SiteMap Utility for DNN Blog Module: Blog SiteMap Utility v01.00.01: This is the first public release of the SiteMap Utility for the core DotNetNuke® Blog Module. Please see the documentation on this site on how to...SqlDiffFramework-A Visual Differencing Engine for Dissimilar Data Sources: SqlDiffFramework 1.0.2.0: Maintenance Release Defect Fixes: Issue # 3: 3 Issue # 4: 4 Enhancements: About Box now displays regional and language settings in effect. SDF...SuperSocket: First release of SuperSocket: !First release of SuperSocketThe Fastcopy Helper: FastcopyHelper: Fastcopy Helper 2.0 This is a final one. You can use it on the way. In order to use it , you should have the .NET3.5 ! 此软件必须下载 .NET3.5平台,方可使用!TV Show Renamer: TV Show Renamer Beta 3: I found the bug the prevented it from closing correctly so I fixed it and had to release it right away. If anyone else finds any problems. contact me.UrzaGatherer: UrzaGatherer v2.0: UrzaGatherer is the first stable version. This release include UrzaBackgroundPictures.VisualStudio 2010 JavaScript Outlining: VisualStudion 2010 Javascript Outlining 1.0: Features Outlines JavaScript codeblock regions for the code placed between { }. Both places on a new line. Outlines custom regions defined by: //...Wouter's SharePoint Demo Land: Navigation Service with Proxy: A SharePoint 2010 Service Application that uses service proxies to relay commands to the actual service. The demo proxy makes use of in-memory comm...盘古分词-开源中文分词组件: V2.0.0.0: 进一步优化性能,分词速度达到将近 500K ,1.2.0.1 版本只有 320K 修改 PanGu.Lucene.Analyzer, 支持 Lucene.net 2.9 版本。 增加对字典中以数字开头的专业非中文词汇的识别 增加英文分词开关,权重由英文小写权重和英文词根权重两个参数来决定...Most Popular ProjectsCommunity Forums NNTP bridgeOutSyncASP.NET MVC Time PlannerNeatUploadMoonyDesk (windows desktop widgets)AgUnit - Silverlight unit testing with ReSharperViperWorks IgnitionASP.NET MVC ExtensionsAviva Solutions C# Coding GuidelinesMute4Most Active ProjectsCommunity Forums NNTP bridgeRawrpatterns & practices – Enterprise LibraryIonics Isapi Rewrite FilterGMap.NET - Great Maps for Windows Forms & PresentationN2 CMSStyleCopFarseer Physics Enginesmark C# LibraryMirror Testing System

    Read the article

  • CodePlex Daily Summary for Sunday, April 15, 2012

    CodePlex Daily Summary for Sunday, April 15, 2012Popular ReleasesAssaultCube Reloaded: 2.4.1 Valor: POSSIBLE KNIFE CRASH FIX Codename Valor as suggested by LMFAO! on the forums Weapon tweaks Linux has Ubuntu 11.10 32-bit precompiled binaries and Ubuntu 10.10 64-bit precompiled binaries, but you can compile your own as it also contains the source. If you are using Mac or other operating systems, download the Linux package. The server pack is ready for both Windows and Linux, but you might need to compile your own for linux (source included)callisto: callisto 2.0.25: removed 2 ip ranges from hotspot shield black list.KBCsv: KBCsv V1.4.0.0: #11872 (skipping records with delimited text can break parser) #11873 (globalization support in CsvWriter) #11185 (more versatile constructor and method overloads)National Geographic Photo of the Day Wallpaper Changer: Photo of the Day Wallpaper Changer v2.0: National Geographic - Photo of the Day Wallpaper Changer v2.0 is an improved version. It has some new features like improved GUI, automatic update and date to date photo archiver etc. please check out the user guide for more information. Please copy the exe in a directory and run. Its that simple to use :).Coding4Fun Tools: Coding4Fun.Phone.Toolkit v1.5.6: Bug Fixes nuget was broken for Timespan and complete build due to how i did the target. Corrected and made all 3 match. Color Slider (and by default Color Picker) didn't respect view state for being disabled depending on how it was set. Test application now has test cases.Json.NET: Json.NET 4.5 Release 3: Change - DefaultContractResolver.IgnoreSerializableAttribute is now true by default Fix - Fixed MaxDepth on JsonReader recursively throwing an error Fix - Fixed SerializationBinder.BindToName not being called with full assembly namesVisual Studio Team Foundation Server Branching and Merging Guide: v2 - For Visual Studio 11: Welcome to the BETA of the Branching and Merging Guide preview As this is a BETA release and the quality bar for the final Release has not been achieved, we value your candid feedback and recommend that you do not use or deploy these BETA artifacts in a production environment. Quality-Bar Details Documentation has been reviewed by Visual Studio ALM Rangers Documentation has not been through an independent technical review Documentation has been reviewed by the quality and recording te...Media Companion: MC 3.435b Release: This release should be the last beta for 3.4xx. A handful of problems have been sorted out since last weeks release. If there are no major problems this time, it will upgraded to 3.500 Stable at the end of the week! General The .NET Framework has been modified to use the Client profile, as provided by normal Windows updates; no longer is there a requirement to download and install the Full profile! mc_com.exe has been worked on to mimic proper Media Companion output (a big thanks to vbat99...Wholemy.LinkedLists: wholemy.linkedlists.2012.04.12.38: libs and srcTHE NVL Maker: The NVL Maker Ver 3.12: SIM??????,TRA??????,ZIP????。 ????????????????,??????~(??????????????????) ??????? simpatch1440x900 trapatch1440x900 ?????1400x900??1440x900,?????????????Data.xp3。 ???? ?????3.12?EXE????????????????, ??????????????,??Tool/krkrconf.exe,??Editor.exe, ???????????????「??????」。 ?????Editor.exe??????。 ???? ???? http://etale.us/gameupload/THE_NVL_Maker_ver3.12_sim.zip ???? http://www.mediafire.com/?je51683g22bz8vo ??Infinite Creation?? http://bbs.etale.us/forum.php ?????? ???? 3.12 ??? ???、????...Quick Performance Monitor: Version 1.8.2: Version 1.8.2. Add the ability for qpmset files to also store the Window location/size so predefined 'sets' can be forced to always open on the same place of the screen.SnmpMessenger: 0.1.1.1: Project Description SnmpMessenger, a messenger. Using the SNMP protocol to exchange messages. It's developed in C#. SnmpMessenger For .Net 4.0, Mono 2.8. Support SNMP V1, V2, V3. Features Send get, set and other requests and get the response. Send and receive traps. Handle requests and return the response. Note This library is compliant with the Common Language Specification(CLS). The latest version is 0.1.1.1. It is only a messenger, does not involve VACM. Any problems, Please mailto: wa...Python Tools for Visual Studio: 1.1.1: We’re pleased to announce the release of Python Tools for Visual Studio 1.1.1. Python Tools for Visual Studio (PTVS) is an open-source plug-in for Visual Studio which supports programming with the Python language. PTVS supports a broad range of features including: • Supports CPython and IronPython • Python editor with advanced member and signature intellisense • Code navigation: “Find all refs”, goto definition, and object browser • Local and remote debugging • Profiling with multiple view...Supporting Guidance and Whitepapers: v1 - Team Foundation Service Whitepapers: Welcome to the BETA release of the Team Foundation Service Whitepapers preview As this is a BETA release and the quality bar for the final Release has not been achieved, we value your candid feedback and recommend that you do not use or deploy these BETA artifacts in a production environment. Quality-Bar Details Documentation has been reviewed by Visual Studio ALM Rangers Documentation has been through an independent technical review All critical bugs have been resolved Known Issue...LINQ to Twitter: LINQ to Twitter Beta v2.0.24: Supports .NET 3.5, .NET 4.0, Silverlight 4.0, Windows Phone 7.1, and Client Profile. 100% Twitter API coverage. Also available via NuGet.Kendo UI ASP.NET Sample Applications: Sample Applications (2012-04-11): Sample application(s) demonstrating the use of Kendo UI in ASP.NET applications.SCCM Client Actions Tool: SCCM Client Actions Tool v1.12: SCCM Client Actions Tool v1.12 is the latest version. It comes with following changes since last version: Improved WMI date conversion to be aware of timezone differences and DST. Fixed new version check. The tool is downloadable as a ZIP file that contains four files: ClientActionsTool.hta – The tool itself. Cmdkey.exe – command line tool for managing cached credentials. This is needed for alternate credentials feature when running the HTA on Windows XP. Cmdkey.exe is natively availab...Dual Browsing: Dual Browser: Please note the following: I setup the address bar temporarily to only accepts http:// .com addresses. Just type in the name of the website excluding: http://, www., and .com; (Ex: for www.youtube.com just type: youtube then click OK). The page splitter can be grabbed by holding down your left mouse button and move left or right. By right clicking on the page background, you can choose to refresh, go back a page and so on. Demo video: http://youtu.be/L7NTFVM3JUYLiberty: v3.2.0.1 Release 9th April 2012: Change Log-Fixed -Reach Fixed a bug where the object editor did not work on non-English operating systemsPath Copy Copy: 10.1: This release addresses the following work items: 11357 11358 11359 This release is a recommended upgrade, especially for users who didn't install the 10.0.1 version.New ProjectsADENA: This project consists in the development of a Graphical User Interface (GUI) for a proof assistant for Propositional Classical Logic based on Natural Deduction method.Alternity Warships Editor: This is a tool designed to easy the process of creating a new spaceship for Alternity using the Warship rules.BigBallz: Projeto de site de Bolões para campeonatos diversos. A princípio pensado para copa do mundo de futebol de 2010Crescent: bunch of mac scriptsDemoVasquez: Demo Proyecto 1Didrotuos: DidrotuosEat Out Advocate: Eat Out Advocate --------------------Kuttiflow: A simple implementation of Workflow for .NetLuan Van Cuoi Khoa: Lu?n van cu?i khoa ÐH Tây Nguyênmapaconwindowsphone: probar un mapa de bing mapsMovie Renamer: Is your movie collection a mess? No idea which movie is which? This tool easily renames movies based on the title and the year of the movie. Helps you sort out that movie collection! It will download from IMDB the closest matches based on the name of the file. Very simple written in WPF, so it will need the .NET 4 framework. At the moment you cannot set how you want to movie to be renamed, it will always be: "MovieTitle (Year).ext".Online Math Calculator: This is an Online Math Calculator. What this does is take your equation in the usual form and convert every variable to x, y, and every constant to a, b, c, d, e, etc... use wolfram|alpha and then convert back to the input format. This allows to input most equations.Orchard on Windows Azure with Dynamic Deploy: Dynamic Deploy is a cloud deployment platform. This project includes the Orchard source code that was used to create a Windows Azure build. The original source has not been modified. We have just added more themes and modules and modified web.config with a machine key. visit httppcvvpes: pcvvpesPesquisa de Satisfacao: PesquisaPit of Despair: An XNA 4.0 game in C# focused on learning to write overhead dungeon crawl games. Inpired from games such as Zelda, Wizardy, perhaps some original Final Fantasy.Prova Branquinho: Prova BranquinhoRibHat: RibHat is a framework for building websites, forums, blogs, and web-based information systems. It is a set of libraries that help the programmer to get rid from the immobility of CMS, obtaining a maximum level of customization.SetupWizard: a SetupWizard, install windows service, create IIS site, create database during installation.SharkOS: This operating system is the system Arkadia OS but with a GUI, it is created in c #, it will be fast and no lag.Shopping List: Shopping List is a simple WP7 application that enables tracking of items to buy when going for groceries.Sim Cricket: Cricket simulation game.. For cricket and C# fans.. Simple InterNET Daemon: Simple InterNET DaemonSteggy: Steganography project. SychevIgor Win8 Apps Source Code: SychevIgor Win8 Apps Source Codetest53768492156478: nothingTool to change monitor display frequency on HTPC: This is a small application that can change the monitors refresh-rate by simply running one of the applications for the desired refresh-rate. It was developed to make it easy to change the refresh rate, when launching external media players from XBMC and so on... And it was published here, so that others can see how easily it can be done.TravelSaver: Hajj Umrah USA, Travel SaverUpdateBot: UpdateBot is a GUI application that simplifies and automates the downloading and parsing of FileHippo.com's Update Checker result pages.

    Read the article

  • So…is it a Seek or a Scan?

    - by Paul White
    You’re probably most familiar with the terms ‘Seek’ and ‘Scan’ from the graphical plans produced by SQL Server Management Studio (SSMS).  The image to the left shows the most common ones, with the three types of scan at the top, followed by four types of seek.  You might look to the SSMS tool-tip descriptions to explain the differences between them: Not hugely helpful are they?  Both mention scans and ranges (nothing about seeks) and the Index Seek description implies that it will not scan the index entirely (which isn’t necessarily true). Recall also yesterday’s post where we saw two Clustered Index Seek operations doing very different things.  The first Seek performed 63 single-row seeking operations; and the second performed a ‘Range Scan’ (more on those later in this post).  I hope you agree that those were two very different operations, and perhaps you are wondering why there aren’t different graphical plan icons for Range Scans and Seeks?  I have often wondered about that, and the first person to mention it after yesterday’s post was Erin Stellato (twitter | blog): Before we go on to make sense of all this, let’s look at another example of how SQL Server confusingly mixes the terms ‘Scan’ and ‘Seek’ in different contexts.  The diagram below shows a very simple heap table with two columns, one of which is the non-clustered Primary Key, and the other has a non-unique non-clustered index defined on it.  The right hand side of the diagram shows a simple query, it’s associated query plan, and a couple of extracts from the SSMS tool-tip and Properties windows. Notice the ‘scan direction’ entry in the Properties window snippet.  Is this a seek or a scan?  The different references to Scans and Seeks are even more pronounced in the XML plan output that the graphical plan is based on.  This fragment is what lies behind the single Index Seek icon shown above: You’ll find the same confusing references to Seeks and Scans throughout the product and its documentation. Making Sense of Seeks Let’s forget all about scans for a moment, and think purely about seeks.  Loosely speaking, a seek is the process of navigating an index B-tree to find a particular index record, most often at the leaf level.  A seek starts at the root and navigates down through the levels of the index to find the point of interest: Singleton Lookups The simplest sort of seek predicate performs this traversal to find (at most) a single record.  This is the case when we search for a single value using a unique index and an equality predicate.  It should be readily apparent that this type of search will either find one record, or none at all.  This operation is known as a singleton lookup.  Given the example table from before, the following query is an example of a singleton lookup seek: Sadly, there’s nothing in the graphical plan or XML output to show that this is a singleton lookup – you have to infer it from the fact that this is a single-value equality seek on a unique index.  The other common examples of a singleton lookup are bookmark lookups – both the RID and Key Lookup forms are singleton lookups (an RID lookup finds a single record in a heap from the unique row locator, and a Key Lookup does much the same thing on a clustered table).  If you happen to run your query with STATISTICS IO ON, you will notice that ‘Scan Count’ is always zero for a singleton lookup. Range Scans The other type of seek predicate is a ‘seek plus range scan’, which I will refer to simply as a range scan.  The seek operation makes an initial descent into the index structure to find the first leaf row that qualifies, and then performs a range scan (either backwards or forwards in the index) until it reaches the end of the scan range. The ability of a range scan to proceed in either direction comes about because index pages at the same level are connected by a doubly-linked list – each page has a pointer to the previous page (in logical key order) as well as a pointer to the following page.  The doubly-linked list is represented by the green and red dotted arrows in the index diagram presented earlier.  One subtle (but important) point is that the notion of a ‘forward’ or ‘backward’ scan applies to the logical key order defined when the index was built.  In the present case, the non-clustered primary key index was created as follows: CREATE TABLE dbo.Example ( key_col INTEGER NOT NULL, data INTEGER NOT NULL, CONSTRAINT [PK dbo.Example key_col] PRIMARY KEY NONCLUSTERED (key_col ASC) ) ; Notice that the primary key index specifies an ascending sort order for the single key column.  This means that a forward scan of the index will retrieve keys in ascending order, while a backward scan would retrieve keys in descending key order.  If the index had been created instead on key_col DESC, a forward scan would retrieve keys in descending order, and a backward scan would return keys in ascending order. A range scan seek predicate may have a Start condition, an End condition, or both.  Where one is missing, the scan starts (or ends) at one extreme end of the index, depending on the scan direction.  Some examples might help clarify that: the following diagram shows four queries, each of which performs a single seek against a column holding every integer from 1 to 100 inclusive.  The results from each query are shown in the blue columns, and relevant attributes from the Properties window appear on the right: Query 1 specifies that all key_col values less than 5 should be returned in ascending order.  The query plan achieves this by seeking to the start of the index leaf (there is no explicit starting value) and scanning forward until the End condition (key_col < 5) is no longer satisfied (SQL Server knows it can stop looking as soon as it finds a key_col value that isn’t less than 5 because all later index entries are guaranteed to sort higher). Query 2 asks for key_col values greater than 95, in descending order.  SQL Server returns these results by seeking to the end of the index, and scanning backwards (in descending key order) until it comes across a row that isn’t greater than 95.  Sharp-eyed readers may notice that the end-of-scan condition is shown as a Start range value.  This is a bug in the XML show plan which bubbles up to the Properties window – when a backward scan is performed, the roles of the Start and End values are reversed, but the plan does not reflect that.  Oh well. Query 3 looks for key_col values that are greater than or equal to 10, and less than 15, in ascending order.  This time, SQL Server seeks to the first index record that matches the Start condition (key_col >= 10) and then scans forward through the leaf pages until the End condition (key_col < 15) is no longer met. Query 4 performs much the same sort of operation as Query 3, but requests the output in descending order.  Again, we have to mentally reverse the Start and End conditions because of the bug, but otherwise the process is the same as always: SQL Server finds the highest-sorting record that meets the condition ‘key_col < 25’ and scans backward until ‘key_col >= 20’ is no longer true. One final point to note: seek operations always have the Ordered: True attribute.  This means that the operator always produces rows in a sorted order, either ascending or descending depending on how the index was defined, and whether the scan part of the operation is forward or backward.  You cannot rely on this sort order in your queries of course (you must always specify an ORDER BY clause if order is important) but SQL Server can make use of the sort order internally.  In the four queries above, the query optimizer was able to avoid an explicit Sort operator to honour the ORDER BY clause, for example. Multiple Seek Predicates As we saw yesterday, a single index seek plan operator can contain one or more seek predicates.  These seek predicates can either be all singleton seeks or all range scans – SQL Server does not mix them.  For example, you might expect the following query to contain two seek predicates, a singleton seek to find the single record in the unique index where key_col = 10, and a range scan to find the key_col values between 15 and 20: SELECT key_col FROM dbo.Example WHERE key_col = 10 OR key_col BETWEEN 15 AND 20 ORDER BY key_col ASC ; In fact, SQL Server transforms the singleton seek (key_col = 10) to the equivalent range scan, Start:[key_col >= 10], End:[key_col <= 10].  This allows both range scans to be evaluated by a single seek operator.  To be clear, this query results in two range scans: one from 10 to 10, and one from 15 to 20. Final Thoughts That’s it for today – tomorrow we’ll look at monitoring singleton lookups and range scans, and I’ll show you a seek on a heap table. Yes, a seek.  On a heap.  Not an index! If you would like to run the queries in this post for yourself, there’s a script below.  Thanks for reading! IF OBJECT_ID(N'dbo.Example', N'U') IS NOT NULL BEGIN DROP TABLE dbo.Example; END ; -- Test table is a heap -- Non-clustered primary key on 'key_col' CREATE TABLE dbo.Example ( key_col INTEGER NOT NULL, data INTEGER NOT NULL, CONSTRAINT [PK dbo.Example key_col] PRIMARY KEY NONCLUSTERED (key_col) ) ; -- Non-unique non-clustered index on the 'data' column CREATE NONCLUSTERED INDEX [IX dbo.Example data] ON dbo.Example (data) ; -- Add 100 rows INSERT dbo.Example WITH (TABLOCKX) ( key_col, data ) SELECT key_col = V.number, data = V.number FROM master.dbo.spt_values AS V WHERE V.[type] = N'P' AND V.number BETWEEN 1 AND 100 ; -- ================ -- Singleton lookup -- ================ ; -- Single value equality seek in a unique index -- Scan count = 0 when STATISTIS IO is ON -- Check the XML SHOWPLAN SELECT E.key_col FROM dbo.Example AS E WHERE E.key_col = 32 ; -- =========== -- Range Scans -- =========== ; -- Query 1 SELECT E.key_col FROM dbo.Example AS E WHERE E.key_col <= 5 ORDER BY E.key_col ASC ; -- Query 2 SELECT E.key_col FROM dbo.Example AS E WHERE E.key_col > 95 ORDER BY E.key_col DESC ; -- Query 3 SELECT E.key_col FROM dbo.Example AS E WHERE E.key_col >= 10 AND E.key_col < 15 ORDER BY E.key_col ASC ; -- Query 4 SELECT E.key_col FROM dbo.Example AS E WHERE E.key_col >= 20 AND E.key_col < 25 ORDER BY E.key_col DESC ; -- Final query (singleton + range = 2 range scans) SELECT E.key_col FROM dbo.Example AS E WHERE E.key_col = 10 OR E.key_col BETWEEN 15 AND 20 ORDER BY E.key_col ASC ; -- === TIDY UP === DROP TABLE dbo.Example; © 2011 Paul White email: [email protected] twitter: @SQL_Kiwi

    Read the article

  • CodePlex Daily Summary for Tuesday, April 13, 2010

    CodePlex Daily Summary for Tuesday, April 13, 2010New ProjectsChat Neo: Video chatDev-wow HappyFuwa: Silverlight + Asp.net + Ajax 实现的以北京奥运为题材,福娃在线聊天互动系统 登录系统后,你可以和线上的朋友即时互动,走动 聊天 动作等都会呈现给其他的在线用户Dynamic Configuration Manager: Dynamic Configuration Manager GameHelper: the project of myselfGeotron: Geotron is a C# geolocation library to resolve postcodes and addresses to co-ordinates, to assist developers in creating location-aware applications. InfoPath Forms Services 2010 Web Testing Toolkit: This project has the tools and information needed to write Visual Studio web tests for InfoPath Forms Services 2010.IronBrainFuck, SimpleBrainFuck: IronBrainFuck and SimpleBrainFuck makes it easier for BrainFuck programmers to develop BrainFuck-compatible programs. It's developed in C#.Runtime Intelligence API: The Runtime Intelligence API library and samples provided by PreEmptive Solutions.SilverVNC 1.0: This project is a Silverlight VNC Viewer. It requires Silverlight 4.0 and works in Out of Browser with full-trust.Snippet Creator: Yet another Visual Studio plugin for creating code snippets.Software Codex: Software Codex is a collection of projects developed in .net to provide a set of libraries and functionalities for developers. It is divided into m...TestCrm: Let go!Make our CrmVidCoder: VidCoder is a DVD ripping and video transcoding application. It uses HandBrake for the encoding engine, but has a revamped and easy to use UI writt...WPF Data Virtualization: Component for displaying and interacting a large data set in WPF application.WPF Gantt chart: Gantt chart control for WPFNew ReleasesAJAX Control Toolkit: 40412: AJAX Control Toolkit Release Notes - April 12th 2010 Release Version 40412April 12, 2010 release of the AJAX Control Toolkit. AJAX Control Toolkit...ASP.NET MVC | SCAFFOLD: ASP.NET MVC SCAFFOLD 1.0 PREVIEW: Primeiro release do ASP.NET MVC SCAFFOLD.Autenticar no OpenLDAP utilizando pGIna: LDAPAuth plugin: Release: DLL LDAPAuth Brief: pGina pluginBluetooth Radar: Version 1.8: Add position helper class to test whether a given point is on the interior of a circle. Random set of Devices on the radar + Zindex changes on Mous...Database Searcher: DB-Searcher Binaries v0.1: First beta version containing following features: Search exact database values via .NET DB-Provider Microsoft SQL MySQL .NET Connector (no .NET t...DBSourceTools: DBSourceTools_1.2.0.7: Release 1.2.0.7 Extended search engine from (pegas)'s patch. Fixed Script Data bug with reserved word (eripsni). Write Targets can now create targe...ESB Toolkit Extensions: Tellago SOA ESB Extenstions v0.4: Windows Installer file that installs Library on a BizTalk ESB 2.0 system. This Install automatically configures the esb.config to use the new compo...Fluent Assertions: Release 1.2: See this blog post for more details on this release: http://www.dennisdoomen.net/2010/04/fluent-assertions-12-has-been-released.htmlFNA Fractal Numerical Algorithm for a new encryption technology: FNA: This is a latest distribution ( 0.04 at the moment). Is a Perl package (.pm). More information on: http://search.cpan.org/~anak/Free Silverlight & WPF Chart Control - Visifire: Visifire SL and WPF Charts 3.0.6 Released!: Hi, Today we have released the final version of Visifire v3.0.6 which contains the following major features: * Zoom using interactive ZoomRec...HD-Trailers.NET Downloader: HD-Trailers.NET_Downloader_v.91_BETA: - added configuration option 'FeedAddress' to specify the URL of the RSS feed to consume - implemented fix for workitem4260: AddDate = false; will ...HobbyBrew Mobile: Beta 1: First public BetaHome Access Plus+: v3.2.6.1: v3.2.6.1 Fixed the wrong date in the iCal Generator Fixed the admin booking posting logging it as being booked by the admin Fixed the problem o...HTML Ruby: 6.21.1: Added back the space ruby text option More consistent ruby text positioning regardless to the page's stylesInfoPath Forms Services 2010 Web Testing Toolkit: IPFS 2010 Web Test Toolkit 20100412 for VS2008: The ExtractAndSubstituteDynamicInfoPathData web test plugin. To use it, simply add the plugin to your web test. It automatically recognizes the inf...IronPython: 2.6.1: Hello Python Community, We’re pleased to announce the final release of IronPython 2.6.1. This version of IronPython makes great strides in stabili...IronRuby: 1.0: IronRuby 1.0 is the first stable version of IronRuby, targeting Ruby 1.8.6 compatibility. For a high-level compatibility report solely based on Rub...METAR.NET Decoder: 0.3.x beta: First public release. Main of the application is working. Metar can be downloaded, decoded, updated and encoded back to metar string. Release incl...MiniTwitter: 1.11: MiniTwitter 1.11 更新内容 修正 設定ファイルを自動でバックアップして、破損したときは出来るだけ修正するように。 初回起動時にタイムラインを更新しようとすると落ちるバグを修正。MSBuild Mercurial Tasks: 1.0.1 Stable: Ready for Production release. This version integrates all the basic functionalities of Mercurial as defined in the Use Case 1.Multiplayer Quiz: Release 1_7_0_0: Latest Version Strongly recommended to use .NET 4.0 now that it is in RC It can be downloaded from hereMVC Foolproof Validation: Beta 0.9.3754: First Beta release. Addressed several bugs from alpha along with some considerable class refactoring. ModelAwareValidationAttribute will make creat...NodeXL: Network Overview, Discovery and Exploration for Excel: NodeXL Excel 2007 Template, version 1.0.1.121: The NodeXL Excel 2007 template displays a network graph using edge and vertex lists stored in an Excel 2007 workbook. What's NewThis version allow...Proxi [Proxy Interface]: Proxi Release 1.0.0.412: Proxi Release 1.0.0.412QueryUnit: QueryUnitPOC v. 0.0.0.8: This version add support for AreNotEqual, Greater, Less and fix some problems with "format" attribute used in conjunction with the string data type...Rainier: Trabalhos de orçamento empresarial: Estão disponíveis os arquivos de exemplo sobre o planejamento para orçamento empresarial. A resolução é referente aos exercícios explicados em sal...Runtime Intelligence API: Initial release: The initial release of the WCF contract & proxy assembly. AuthentiCode signed library.SharePoint Accelerators: Central Admin - Command Search: This web part allows you to search for a SharePoint 2010 Central Admin commands. This web part can come handy when you are demostrating SharePoint ...SharePoint Labs: SPLab5013A-FRA-Level100: SPLab5013A-FRA-Level100* This SharePoint Lab will teach you how to provision a computed site column that shows a customized view of an existing hid...SilverVNC 1.0: SilverVNC 1.0.3755.0: This download is the first release of the project published on www.silverlightplayground.org. For a detailed explanation please refer to http://www...Snippet Creator: SnippetCreator.Setup: This is the first and (I hope) final release.SQL Server Health & History (SQLH2): SQLH2 v.2.2.001: New Features Updated to use .Net 3.5 Job and Job history information implemented Last dif and log backup columns added Logical Disk implemented Dis...SQL Server Health & History (SQLH2): SQLH2PerfCollector v.2.1.003: Updated to run on .Net 3.5 Now installs to correct registry path on x64Star Trooper for XNA 2D Tutorial: Lesson one content: Here is Lesson one original content for the StarTrooper 2D XNA tutorial. The blog tutorial has now started over on http://xna-uk.net/blogs/darkgen...SysPad: SysPad 4.10.7.1: A folder management and scratchpad utility; especially useful in a business network setting that utilizes numerous, commonly used folders. The pro...TaskUnZip for SSIS: TaskUnZip for SSIS 1.1.0.0: Add: recursive compress. Add: filter option for exstract e compress file. (Tnx to: Kevin Wendler)TCP Wrapper: TCP Wrapper 1.0.0.3: Adding Client Accessor to CommingDataAvailableEventArgs ...UCD: Architecture: UCDArch 1.0: Production release of UCDArch 1.0 (for ASP.NET MVC 1.0). New Features including the ability to modify the NHibernate FlushMode, URL convention help...VCC: Latest build, v2.1.30412.0: Automatic drop of latest buildVidCoder: 0.1.0: First VidCoder beta release. It's missing a few features that will be added before release: Advanced x264 options In-GUI encode log Additiona...WatchersNET CKEditor™ Provider for DotNetNuke: CKEditor Provider 1.10.01: Whats New changesBrowser: Removed extra "\" sign from Current folder name selecting the root folder Browser: Fixed Folder Rendering Browser Fix...WPF Gantt chart: gantt: first, alpha version, of gantt chart for wpfxvanneste: Coverflow et thumbnail sharepoint: Code du coverflow silverlight du webcast sur les thumbnails sharepointMost Popular ProjectsWBFS ManagerRawrMicrosoft SQL Server Product Samples: DatabaseAJAX Control ToolkitSilverlight ToolkitWindows Presentation Foundation (WPF)ASP.NETMicrosoft SQL Server Community & SamplesFacebook Developer ToolkitPHPExcelMost Active ProjectsRawrnopCommerce. Open Source online shop e-commerce solution.AutoPocopatterns & practices – Enterprise LibraryShweet: SharePoint 2010 Team Messaging built with PexFarseer Physics EngineNB_Store - Free DotNetNuke Ecommerce Catalog ModuleIonics Isapi Rewrite FilterBlogEngine.NETBeanProxy

    Read the article

  • VLOOKUP in Excel, part 2: Using VLOOKUP without a database

    - by Mark Virtue
    In a recent article, we introduced the Excel function called VLOOKUP and explained how it could be used to retrieve information from a database into a cell in a local worksheet.  In that article we mentioned that there were two uses for VLOOKUP, and only one of them dealt with querying databases.  In this article, the second and final in the VLOOKUP series, we examine this other, lesser known use for the VLOOKUP function. If you haven’t already done so, please read the first VLOOKUP article – this article will assume that many of the concepts explained in that article are already known to the reader. When working with databases, VLOOKUP is passed a “unique identifier” that serves to identify which data record we wish to find in the database (e.g. a product code or customer ID).  This unique identifier must exist in the database, otherwise VLOOKUP returns us an error.  In this article, we will examine a way of using VLOOKUP where the identifier doesn’t need to exist in the database at all.  It’s almost as if VLOOKUP can adopt a “near enough is good enough” approach to returning the data we’re looking for.  In certain circumstances, this is exactly what we need. We will illustrate this article with a real-world example – that of calculating the commissions that are generated on a set of sales figures.  We will start with a very simple scenario, and then progressively make it more complex, until the only rational solution to the problem is to use VLOOKUP.  The initial scenario in our fictitious company works like this:  If a salesperson creates more than $30,000 worth of sales in a given year, the commission they earn on those sales is 30%.  Otherwise their commission is only 20%.  So far this is a pretty simple worksheet: To use this worksheet, the salesperson enters their sales figures in cell B1, and the formula in cell B2 calculates the correct commission rate they are entitled to receive, which is used in cell B3 to calculate the total commission that the salesperson is owed (which is a simple multiplication of B1 and B2). The cell B2 contains the only interesting part of this worksheet – the formula for deciding which commission rate to use: the one below the threshold of $30,000, or the one above the threshold.  This formula makes use of the Excel function called IF.  For those readers that are not familiar with IF, it works like this: IF(condition,value if true,value if false) Where the condition is an expression that evaluates to either true or false.  In the example above, the condition is the expression B1<B5, which can be read as “Is B1 less than B5?”, or, put another way, “Are the total sales less than the threshold”.  If the answer to this question is “yes” (true), then we use the value if true parameter of the function, namely B6 in this case – the commission rate if the sales total was below the threshold.  If the answer to the question is “no” (false), then we use the value if false parameter of the function, namely B7 in this case – the commission rate if the sales total was above the threshold. As you can see, using a sales total of $20,000 gives us a commission rate of 20% in cell B2.  If we enter a value of $40,000, we get a different commission rate: So our spreadsheet is working. Let’s make it more complex.  Let’s introduce a second threshold:  If the salesperson earns more than $40,000, then their commission rate increases to 40%: Easy enough to understand in the real world, but in cell B2 our formula is getting more complex.  If you look closely at the formula, you’ll see that the third parameter of the original IF function (the value if false) is now an entire IF function in its own right.  This is called a nested function (a function within a function).  It’s perfectly valid in Excel (it even works!), but it’s harder to read and understand. We’re not going to go into the nuts and bolts of how and why this works, nor will we examine the nuances of nested functions.  This is a tutorial on VLOOKUP, not on Excel in general. Anyway, it gets worse!  What about when we decide that if they earn more than $50,000 then they’re entitled to 50% commission, and if they earn more than $60,000 then they’re entitled to 60% commission? Now the formula in cell B2, while correct, has become virtually unreadable.  No-one should have to write formulae where the functions are nested four levels deep!  Surely there must be a simpler way? There certainly is.  VLOOKUP to the rescue! Let’s redesign the worksheet a bit.  We’ll keep all the same figures, but organize it in a new way, a more tabular way: Take a moment and verify for yourself that the new Rate Table works exactly the same as the series of thresholds above. Conceptually, what we’re about to do is use VLOOKUP to look up the salesperson’s sales total (from B1) in the rate table and return to us the corresponding commission rate.  Note that the salesperson may have indeed created sales that are not one of the five values in the rate table ($0, $30,000, $40,000, $50,000 or $60,000).  They may have created sales of $34,988.  It’s important to note that $34,988 does not appear in the rate table.  Let’s see if VLOOKUP can solve our problem anyway… We select cell B2 (the location we want to put our formula), and then insert the VLOOKUP function from the Formulas tab: The Function Arguments box for VLOOKUP appears.  We fill in the arguments (parameters) one by one, starting with the Lookup_value, which is, in this case, the sales total from cell B1.  We place the cursor in the Lookup_value field and then click once on cell B1: Next we need to specify to VLOOKUP what table to lookup this data in.  In this example, it’s the rate table, of course.  We place the cursor in the Table_array field, and then highlight the entire rate table – excluding the headings: Next we must specify which column in the table contains the information we want our formula to return to us.  In this case we want the commission rate, which is found in the second column in the table, so we therefore enter a 2 into the Col_index_num field: Finally we enter a value in the Range_lookup field. Important:  It is the use of this field that differentiates the two ways of using VLOOKUP.  To use VLOOKUP with a database, this final parameter, Range_lookup, must always be set to FALSE, but with this other use of VLOOKUP, we must either leave it blank or enter a value of TRUE.  When using VLOOKUP, it is vital that you make the correct choice for this final parameter. To be explicit, we will enter a value of true in the Range_lookup field.  It would also be fine to leave it blank, as this is the default value: We have completed all the parameters.  We now click the OK button, and Excel builds our VLOOKUP formula for us: If we experiment with a few different sales total amounts, we can satisfy ourselves that the formula is working. Conclusion In the “database” version of VLOOKUP, where the Range_lookup parameter is FALSE, the value passed in the first parameter (Lookup_value) must be present in the database.  In other words, we’re looking for an exact match. But in this other use of VLOOKUP, we are not necessarily looking for an exact match.  In this case, “near enough is good enough”.  But what do we mean by “near enough”?  Let’s use an example:  When searching for a commission rate on a sales total of $34,988, our VLOOKUP formula will return us a value of 30%, which is the correct answer.  Why did it choose the row in the table containing 30% ?  What, in fact, does “near enough” mean in this case?  Let’s be precise: When Range_lookup is set to TRUE (or omitted), VLOOKUP will look in column 1 and match the highest value that is not greater than the Lookup_value parameter. It’s also important to note that for this system to work, the table must be sorted in ascending order on column 1! If you would like to practice with VLOOKUP, the sample file illustrated in this article can be downloaded from here. Similar Articles Productive Geek Tips Using VLOOKUP in ExcelImport Microsoft Access Data Into ExcelImport an Access Database into ExcelCopy a Group of Cells in Excel 2007 to the Clipboard as an ImageShare Access Data with Excel in Office 2010 TouchFreeze Alternative in AutoHotkey The Icy Undertow Desktop Windows Home Server – Backup to LAN The Clear & Clean Desktop Use This Bookmarklet to Easily Get Albums Use AutoHotkey to Assign a Hotkey to a Specific Window Latest Software Reviews Tinyhacker Random Tips DVDFab 6 Revo Uninstaller Pro Registry Mechanic 9 for Windows PC Tools Internet Security Suite 2010 Quickly Schedule Meetings With NeedtoMeet Share Flickr Photos On Facebook Automatically Are You Blocked On Gtalk? Find out Discover Latest Android Apps On AppBrain The Ultimate Guide For YouTube Lovers Will it Blend? iPad Edition

    Read the article

< Previous Page | 327 328 329 330 331 332 333 334 335 336 337 338  | Next Page >