Search Results

Search found 37654 results on 1507 pages for 'function prototypes'.

Page 1444/1507 | < Previous Page | 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451  | Next Page >

• SQL SERVER – SSIS Look Up Component – Cache Mode – Notes from the Field #028

  - by Pinal Dave

  [Notes from Pinal]: Lots of people think that SSIS is all about arranging various operations together in one logical flow. Well, the understanding is absolutely correct, but the implementation of the same is not as easy as it seems. Similarly most of the people think lookup component is just component which does look up for additional information and does not pay much attention to it. Due to the same reason they do not pay attention to the same and eventually get very bad performance. Linchpin People are database coaches and wellness experts for a data driven world. In this 28th episode of the Notes from the Fields series database expert Tim Mitchell (partner at Linchpin People) shares very interesting conversation related to how to write a good lookup component with Cache Mode. In SQL Server Integration Services, the lookup component is one of the most frequently used tools for data validation and completion.  The lookup component is provided as a means to virtually join one set of data to another to validate and/or retrieve missing values.  Properly configured, it is reliable and reasonably fast. Among the many settings available on the lookup component, one of the most critical is the cache mode.  This selection will determine whether and how the distinct lookup values are cached during package execution.  It is critical to know how cache modes affect the result of the lookup and the performance of the package, as choosing the wrong setting can lead to poorly performing packages, and in some cases, incorrect results. Full Cache The full cache mode setting is the default cache mode selection in the SSIS lookup transformation.  Like the name implies, full cache mode will cause the lookup transformation to retrieve and store in SSIS cache the entire set of data from the specified lookup location.  As a result, the data flow in which the lookup transformation resides will not start processing any data buffers until all of the rows from the lookup query have been cached in SSIS. The most commonly used cache mode is the full cache setting, and for good reason.  The full cache setting has the most practical applications, and should be considered the go-to cache setting when dealing with an untested set of data. With a moderately sized set of reference data, a lookup transformation using full cache mode usually performs well.  Full cache mode does not require multiple round trips to the database, since the entire reference result set is cached prior to data flow execution. There are a few potential gotchas to be aware of when using full cache mode.  First, you can see some performance issues – memory pressure in particular – when using full cache mode against large sets of reference data.  If the table you use for the lookup is very large (either deep or wide, or perhaps both), there’s going to be a performance cost associated with retrieving and caching all of that data.  Also, keep in mind that when doing a lookup on character data, full cache mode will always do a case-sensitive (and in some cases, space-sensitive) string comparison even if your database is set to a case-insensitive collation.  This is because the in-memory lookup uses a .NET string comparison (which is case- and space-sensitive) as opposed to a database string comparison (which may be case sensitive, depending on collation).  There’s a relatively easy workaround in which you can use the UPPER() or LOWER() function in the pipeline data and the reference data to ensure that case differences do not impact the success of your lookup operation.  Again, neither of these present a reason to avoid full cache mode, but should be used to determine whether full cache mode should be used in a given situation. Full cache mode is ideally useful when one or all of the following conditions exist: The size of the reference data set is small to moderately sized The size of the pipeline data set (the data you are comparing to the lookup table) is large, is unknown at design time, or is unpredictable Each distinct key value(s) in the pipeline data set is expected to be found multiple times in that set of data Partial Cache When using the partial cache setting, lookup values will still be cached, but only as each distinct value is encountered in the data flow.  Initially, each distinct value will be retrieved individually from the specified source, and then cached.  To be clear, this is a row-by-row lookup for each distinct key value(s). This is a less frequently used cache setting because it addresses a narrower set of scenarios.  Because each distinct key value(s) combination requires a relational round trip to the lookup source, performance can be an issue, especially with a large pipeline data set to be compared to the lookup data set.  If you have, for example, a million records from your pipeline data source, you have the potential for doing a million lookup queries against your lookup data source (depending on the number of distinct values in the key column(s)).  Therefore, one has to be keenly aware of the expected row count and value distribution of the pipeline data to safely use partial cache mode. Using partial cache mode is ideally suited for the conditions below: The size of the data in the pipeline (more specifically, the number of distinct key column) is relatively small The size of the lookup data is too large to effectively store in cache The lookup source is well indexed to allow for fast retrieval of row-by-row values No Cache As you might guess, selecting no cache mode will not add any values to the lookup cache in SSIS.  As a result, every single row in the pipeline data set will require a query against the lookup source.  Since no data is cached, it is possible to save a small amount of overhead in SSIS memory in cases where key values are not reused.  In the real world, I don’t see a lot of use of the no cache setting, but I can imagine some edge cases where it might be useful. As such, it’s critical to know your data before choosing this option.  Obviously, performance will be an issue with anything other than small sets of data, as the no cache setting requires row-by-row processing of all of the data in the pipeline. I would recommend considering the no cache mode only when all of the below conditions are true: The reference data set is too large to reasonably be loaded into SSIS memory The pipeline data set is small and is not expected to grow There are expected to be very few or no duplicates of the key values(s) in the pipeline data set (i.e., there would be no benefit from caching these values) Conclusion The cache mode, an often-overlooked setting on the SSIS lookup component, represents an important design decision in your SSIS data flow.  Choosing the right lookup cache mode directly impacts the fidelity of your results and the performance of package execution.  Know how this selection impacts your ETL loads, and you’ll end up with more reliable, faster packages. If you want me to take a look at your server and its settings, or if your server is facing any issue we can Fix Your SQL Server. Reference: Pinal Dave (http://blog.sqlauthority.com)Filed under: Notes from the Field, PostADay, SQL, SQL Authority, SQL Query, SQL Server, SQL Tips and Tricks, T SQL Tagged: SSIS

  Read the article

• More Fun with C# Iterators and Generators

  - by James Michael Hare

  In my last post, I talked quite a bit about iterators and how they can be really powerful tools for filtering a list of items down to a subset of items.  This had both pros and cons over returning a full collection, which, in summary, were:   Pros: If traversal is only partial, does not have to visit rest of collection. If evaluation method is costly, only incurs that cost on elements visited. Adds little to no garbage collection pressure.    Cons: Very slight performance impact if you know caller will always consume all items in collection. And as we saw in the last post, that con for the cost was very, very small and only really became evident on very tight loops consuming very large lists completely.    One of the key items to note, though, is the garbage!  In the traditional (return a new collection) method, if you have a 1,000,000 element collection, and wish to transform or filter it in some way, you have to allocate space for that copy of the collection.  That is, say you have a collection of 1,000,000 items and you want to double every item in the collection.  Well, that means you have to allocate a collection to hold those 1,000,000 items to return, which is a lot especially if you are just going to use it once and toss it.   Iterators, though, don't have this problem.  Each time you visit the node, it would return the doubled value of the node (in this example) and not allocate a second collection of 1,000,000 doubled items.  Do you see the distinction?  In both cases, we're consuming 1,000,000 items.  But in one case we pass back each doubled item which is just an int (for example's sake) on the stack and in the other case, we allocate a list containing 1,000,000 items which then must be garbage collected.   So iterators in C# are pretty cool, eh?  Well, here's one more thing a C# iterator can do that a traditional "return a new collection" transformation can't!   It can return **unbounded** collections!   I know, I know, that smells a lot like an infinite loop, eh?  Yes and no.  Basically, you're relying on the caller to put the bounds on the list, and as long as the caller doesn't you keep going.  Consider this example:   public static class Fibonacci {     // returns the infinite fibonacci sequence     public static IEnumerable<int> Sequence()     {         int iteration = 0;         int first = 1;         int second = 1;         int current = 0;         while (true)         {             if (iteration++ < 2)             {                 current = 1;             }             else             {                 current = first + second;                 second = first;                 first = current;             }             yield return current;         }     } }   Whoa, you say!  Yes, that's an infinite loop!  What the heck is going on there?  Yes, that was intentional.  Would it be better to have a fibonacci sequence that returns only a specific number of items?  Perhaps, but that wouldn't give you the power to defer the execution to the caller.   The beauty of this function is it is as infinite as the sequence itself!  The fibonacci sequence is unbounded, and so is this method.  It will continue to return fibonacci numbers for as long as you ask for them.  Now that's not something you can do with a traditional method that would return a collection of ints representing each number.  In that case you would eventually run out of memory as you got to higher and higher numbers.  This method, though, never runs out of memory.   Now, that said, you do have to know when you use it that it is an infinite collection and bound it appropriately.  Fortunately, Linq provides a lot of these extension methods for you!   Let's say you only want the first 10 fibonacci numbers:       foreach(var fib in Fibonacci.Sequence().Take(10))     {         Console.WriteLine(fib);     }   Or let's say you only want the fibonacci numbers that are less than 100:       foreach(var fib in Fibonacci.Sequence().TakeWhile(f => f < 100))     {         Console.WriteLine(fib);     }   So, you see, one of the nice things about iterators is their power to work with virtually any size (even infinite) collections without adding the garbage collection overhead of making new collections.    You can also do fun things like this to make a more "fluent" interface for for loops:   // A set of integer generator extension methods public static class IntExtensions {     // Begins counting to inifity, use To() to range this.     public static IEnumerable<int> Every(this int start)     {         // deliberately avoiding condition because keeps going         // to infinity for as long as values are pulled.         for (var i = start; ; ++i)         {             yield return i;         }     }     // Begins counting to infinity by the given step value, use To() to     public static IEnumerable<int> Every(this int start, int byEvery)     {         // deliberately avoiding condition because keeps going         // to infinity for as long as values are pulled.         for (var i = start; ; i += byEvery)         {             yield return i;         }     }     // Begins counting to inifity, use To() to range this.     public static IEnumerable<int> To(this int start, int end)     {         for (var i = start; i <= end; ++i)         {             yield return i;         }     }     // Ranges the count by specifying the upper range of the count.     public static IEnumerable<int> To(this IEnumerable<int> collection, int end)     {         return collection.TakeWhile(item => item <= end);     } }   Note that there are two versions of each method.  One that starts with an int and one that starts with an IEnumerable<int>.  This is to allow more power in chaining from either an existing collection or from an int.  This lets you do things like:   // count from 1 to 30 foreach(var i in 1.To(30)) {     Console.WriteLine(i); }     // count from 1 to 10 by 2s foreach(var i in 0.Every(2).To(10)) {     Console.WriteLine(i); }     // or, if you want an infinite sequence counting by 5s until something inside breaks you out... foreach(var i in 0.Every(5)) {     if (someCondition)     {         break;     }     ... }     Yes, those are kinda play functions and not particularly useful, but they show some of the power of generators and extension methods to form a fluid interface.   So what do you think?  What are some of your favorite generators and iterators?

  Read the article

• Design for complex ATG applications

  - by Glen Borkowski

  Overview Needless to say, some ATG applications are more complex than others.  Some ATG applications support a single site, single language, single catalog, single currency, have a single development staff, single business team, and a relatively simple business model.  The real complex applications have to support multiple sites, multiple languages, multiple catalogs, multiple currencies, a couple different development teams, multiple business teams, and a highly complex business model (and processes to go along with it).  While it's still important to implement a proper design for simple applications, it's absolutely critical to do this for the complex applications.  Why?  It's all about time and money.  If you are unable to manage your complex applications in an efficient manner, the cost of managing it will increase dramatically as will the time to get things done (time to market).  On the positive side, your competition is most likely in the same situation, so you just need to be more efficient than they are. This article is intended to discuss a number of key areas to think about when designing complex applications on ATG.  Some of this can get fairly technical, so it may help to get some background first.  You can get enough of the required background information from this post.  After reading that, come back here and follow along. Application Design Of all the various types of ATG applications out there, the most complex tend to be the ones in the telecommunications industry - especially the ones which operate in multiple countries.  To get started, let's assume that we are talking about an application like that.  One that has these properties: Operates in multiple countries - must support multiple sites, catalogs, languages, and currencies The organization is fairly loosely-coupled - single brand, but different businesses across different countries There is some common functionality across all sites in all countries There is some common functionality across different sites within the same country Sites within a single country may have some unique functionality - relative to other sites in the same country Complex product catalog (mostly in terms of bundles, eligibility, and compatibility) At this point, I'll assume you have read through the required reading and have a decent understanding of how ATG modules work... Code / configuration - assemble into modules When it comes to defining your modules for a complex application, there are a number of goals: Divide functionality between the modules in a way that maps to your business Group common functionality 'further down in the stack of modules' Provide a good balance between shared resources and autonomy for countries / sites Now I'll describe a high level approach to how you could accomplish those goals...  Let's start from the bottom and work our way up.  At the very bottom, you have the modules that ship with ATG - the 'out of the box' stuff.  You want to make sure that you are leveraging all the modules that make sense in order to get the most value from ATG as possible - and less stuff you'll have to write yourself.  On top of the ATG modules, you should create what we'll refer to as the Corporate Foundation Module described as follows: Sits directly on top of ATG modules Used by all applications across all countries and sites - this is the foundation for everyone Contains everything that is common across all countries / all sites Once established and settled, will change less frequently than other 'higher' modules Encapsulates as many enterprise-wide integrations as possible Will provide means of code sharing therefore less development / testing - faster time to market Contains a 'reference' web application (described below) The next layer up could be multiple modules for each country (you could replace this with region if that makes more sense).  We'll define those modules as follows: Sits on top of the corporate foundation module Contains what is unique to all sites in a given country Responsible for managing any resource bundles for this country (to handle multiple languages) Overrides / replaces corporate integration points with any country-specific ones Finally, we will define what should be a fairly 'thin' (in terms of functionality) set of modules for each site as follows: Sits on top of the country it resides in module Contains what is unique for a given site within a given country Will mostly contain configuration, but could also define some unique functionality as well Contains one or more web applications The graphic below should help to indicate how these modules fit together: Web applications As described in the previous section, there are many opportunities for sharing (minimizing costs) as it relates to the code and configuration aspects of ATG modules.  Web applications are also contained within ATG modules, however, sharing web applications can be a bit more difficult because this is what the end customer actually sees, and since each site may have some degree of unique look & feel, sharing becomes more challenging.  One approach that can help is to define a 'reference' web application at the corporate foundation layer to act as a solid starting point for each site.  Here's a description of the 'reference' web application: Contains minimal / sample reference styling as this will mostly be addressed at the site level web app Focus on functionality - ensure that core functionality is revealed via this web application Each individual site can use this as a starting point There may be multiple types of web apps (i.e. B2C, B2B, etc) There are some techniques to share web application assets - i.e. multiple web applications, defined in the web.xml, and it's worth investigating, but is out of scope here. Reference infrastructure In this complex environment, it is assumed that there is not a single infrastructure for all countries and all sites.  It's more likely that different countries (or regions) could have their own solution for infrastructure.  In this case, it will be advantageous to define a reference infrastructure which contains all the hardware and software that make up the core environment.  Specifications and diagrams should be created to outline what this reference infrastructure looks like, as well as it's baseline cost and the incremental cost to scale up with volume.  Having some consistency in terms of infrastructure will save time and money as new countries / sites come online.  Here are some properties of the reference infrastructure: Standardized approach to setup of hardware Type and number of servers Defines application server, operating system, database, etc... - including vendor and specific versions Consistent naming conventions Provides a consistent base of terminology and understanding across environments Defines which ATG services run on which servers Production Staging BCC / Preview Each site can change as required to meet scale requirements Governance / organization It should be no surprise that the complex application we're talking about is backed by an equally complex organization.  One of the more challenging aspects of efficiently managing a series of complex applications is to ensure the proper level of governance and organization.  Here are some ideas and goals to work towards: Establish a committee to make enterprise-wide decisions that affect all sites Representation should be evenly distributed Should have a clear communication procedure Focus on high level business goals Evaluation of feature / function gaps and how that relates to ATG release schedule / roadmap Determine when to upgrade & ensure value will be realized Determine how to manage various levels of modules Who is responsible for maintaining corporate / country / site layers Determine a procedure for controlling what goes in the corporate foundation module Standardize on source code control, database, hardware, OS versions, J2EE app servers, development procedures, etc only use tested / proven versions - this is something that should be centralized so that every country / site does not have to worry about compatibility between versions Create a innovation team Quickly develop new features, perform proof of concepts All teams can benefit from their findings Summary At this point, it should be clear why the topics above (design, governance, organization, etc) are critical to being able to efficiently manage a complex application.  To summarize, it's all about competitive advantage...  You will need to reduce costs and improve time to market with the goal of providing a better experience for your end customers.  You can reduce cost by reducing development time, time allocated to testing (don't have to test the corporate foundation module over and over again - do it once), and optimizing operations.  With an efficient design, you can improve your time to market and your business will be more flexible  and agile.  Over time, you'll find that you're becoming more focused on offering functionality that is new to the market (creativity) and this will be rewarded - you're now a leader. In addition to the above, you'll realize soft benefits as well.  Your staff will be operating in a culture based on sharing.  You'll want to reward efforts to improve and enhance the foundation as this will benefit everyone.  This culture will inspire innovation, which can only lend itself to your competitive advantage.

  Read the article

• Issues loading SWF with External SWF Files - using SWFObject and ExternalInterface

  - by wdense51

  I'm having issues with loading a SWF that references external SWF files... The main SWF loads fine if the HTML file is in the same folder as all the SWFs using the following code: <script type="text/javascript" src="../js/swfobject.js"></script> <script type="text/javascript"> var flashvars = {}; var params = { allowScriptAccess: "always" }; params.quality = "high"; params.wmode = "transparent"; var attributes = {id:"IDofSWF", name:"IDofSWF"}; swfobject.embedSWF("event_so_js.swf", "flashContent", "700", "400", "7.0.0", false, flashvars, params, attributes);</script> </head> <body> <div id="flashContent"> <object data="event_so_js.swf" name="IDofSWF" id="IDofSWF" type="application/x-shockwave-flash" width="700" height="400"></object></div> But as soon as I move the HTML file out of that folder to the root folder and update the links, it doesn't load correctly - it seems that it's having trouble with the external SWF files. I did have it successfully load one of the external SWF files directly, but it's having trouble with the main SWF. All of the SWF files are in the same folder, so I don't know why it's having issues. Here's the code for the HTML file when it's in the root folder: <script type="text/javascript" src="js/swfobject.js"></script> <script type="text/javascript"> var flashvars = {}; var params = { allowScriptAccess: "always" }; params.quality = "high"; params.wmode = "transparent"; var attributes = {id:"IDofSWF", name:"IDofSWF"}; swfobject.embedSWF("folio/event_so_js.swf", "flashContent", "700", "400", "9.0.0", false, flashvars, params, attributes);</script> </head> <body> <div id="flashContent"> <object data="folio/event_so_js.swf" name="IDofSWF" id="IDofSWF" type="application/x-shockwave-flash" width="700" height="400"></object></div> There is also a link on the page that calls a function in the actionscript using ExternalInterface, so it could be that causing the issues. The code for the link is: <a href="#" onclick="document.getElementById('IDofSWF').clicky()"> Any help would be awesome, because it's really confusing me.

  Read the article

• Unable to capture standard output of process using Boost.Process

  - by Chris Kaminski

  Currently am using Boost.Process from the Boost sandbox, and am having issues getting it to capture my standard output properly; wondering if someone can give me a second pair of eyeballs into what I might be doing wrong. I'm trying to take thumbnails out of RAW camera images using DCRAW (latest version), and capture them for conversion to QT QImage's. The process launch function: namespace bf = ::boost::filesystem; namespace bp = ::boost::process; QImage DCRawInterface::convertRawImage(string path) { // commandline: dcraw -e -c <srcfile> -> piped to stdout. if ( bf::exists( path ) ) { std::string exec = "bin\\dcraw.exe"; std::vector<std::string> args; args.push_back("-v"); args.push_back("-c"); args.push_back("-e"); args.push_back(path); bp::context ctx; ctx.stdout_behavior = bp::capture_stream(); bp::child c = bp::launch(exec, args, ctx); bp::pistream &is = c.get_stdout(); ofstream output("C:\\temp\\testcfk.jpg"); streamcopy(is, output); } return (NULL); } inline void streamcopy(std::istream& input, std::ostream& out) { char buffer[4096]; int i = 0; while (!input.eof() ) { memset(buffer, 0, sizeof(buffer)); int bytes = input.readsome(buffer, sizeof buffer); out.write(buffer, bytes); i++; } } Invoking the converter: DCRawInterface DcRaw; DcRaw.convertRawImage("test/CFK_2439.NEF"); The goal is to simply verify that I can copy the input stream to an output file. Currently, if I comment out the following line: args.push_back("-c"); then the thumbnail is written by DCRAW to the source directory with a name of CFK_2439.thumb.jpg, which proves to me that the process is getting invoked with the right arguments. What's not happening is connecting to the output pipe properly. FWIW: I'm performing this test on Windows XP under Eclipse 3.5/Latest MingW (GCC 4.4).

  Read the article

• PyGTK: dynamic label wrapping

  - by detly

  It's a known bug/issue that a label in GTK will not dynamically resize when the parent changes. It's one of those really annoying small details, and I want to hack around it if possible. I followed the approach at 16 software, but as per the disclaimer you cannot then resize it smaller. So I attempted a trick mentioned in one of the comments (the set_size_request call in the signal callback), but this results in some sort of infinite loop (try it and see). Does anyone have any other ideas? (You can't block the signal just for the duration of the call, since as the print statements seem to indicate, the problem starts after the function is left.) The code is below. You can see what I mean if you run it and try to resize the window larger and then smaller. (If you want to see the original problem, comment out the line after "Connect to the size-allocate signal", run it, and resize the window bigger.) The Glade file ("example.glade"): <?xml version="1.0"?> <glade-interface> <!-- interface-requires gtk+ 2.16 --> <!-- interface-naming-policy project-wide --> <widget class="GtkWindow" id="window1"> <property name="visible">True</property> <signal name="destroy" handler="on_destroy"/> <child> <widget class="GtkLabel" id="label1"> <property name="visible">True</property> <property name="label" translatable="yes">In publishing and graphic design, lorem ipsum[p][1][2] is the name given to commonly used placeholder text (filler text) to demonstrate the graphic elements of a document or visual presentation, such as font, typography, and layout. The lorem ipsum text, which is typically a nonsensical list of semi-Latin words, is a hacked version of a Latin text by Cicero, with words/letters omitted and others inserted, but not proper Latin[1][2] (see below: History and discovery). The closest English translation would be "pain itself" (dolorem = pain, grief, misery, suffering; ipsum = itself).</property> <property name="wrap">True</property> </widget> </child> </widget> </glade-interface> The Python code: #!/usr/bin/python import pygtk import gobject import gtk.glade def wrapped_label_hack(gtklabel, allocation): print "In wrapped_label_hack" gtklabel.set_size_request(allocation.width, -1) # If you uncomment this, we get INFINITE LOOPING! # gtklabel.set_size_request(-1, -1) print "Leaving wrapped_label_hack" class ExampleGTK: def __init__(self, filename): self.tree = gtk.glade.XML(filename, "window1", "Example") self.id = "window1" self.tree.signal_autoconnect(self) # Connect to the size-allocate signal self.get_widget("label1").connect("size-allocate", wrapped_label_hack) def on_destroy(self, widget): self.close() def get_widget(self, id): return self.tree.get_widget(id) def close(self): window = self.get_widget(self.id) if window is not None: window.destroy() gtk.main_quit() if __name__ == "__main__": window = ExampleGTK("example.glade") gtk.main()

  Read the article

• Django conditional template inheritance

  - by Ed

  I have template that displays object elements with hyperlinks to other parts of my site. I have another function that displays past versions of the same object. In this display, I don't want the hyperlinks. I'm under the assumption that I can't dynamically switch off the hyperlinks, so I've included both versions in the same template. I use an if statement to either display the hyperlinked version or the plain text version. I prefer to keep them in the same template because if I need to change the format of one, it will be easy to apply it to the other right there. The template extends framework.html. Framework has a breadcrumb system and it extends base.html. Base has a simple top menu system. So here's my dilemma. When viewing the standard hyperlink data, I want to see the top menu and the breadcrumbs. But when viewing the past version plain text data, I only want the data, no menu, no breadcrumbs. I'm unsure if this is possible given my current design. I tried having framework inherit the primary template so that I could choose to call either framework (and display the breadcrumbs), or the template itself, thus skipping the breadcrumbs, but I want framework.html available for other templates as well. If framework.html extends a specific template, I lose the ability to display it in other templates. I tried writing an if statement that would display a the top_menu block and the nav_menu block from base.html and framework.html respectively. This would overwrite their blocks and allow me to turn off those elements conditional on the if. Unfortunately, it doesn't appear to be conditional; if the block elements are in the template at all, surrounded by an if or not, I lose the menus. I thought about using {% include %} to pick up the breadcrumbs and a split out top menu. In that case though, I'll have to include it all the time. No more inheritance. Is this the best option given my requirement?

  Read the article

• Scipy Negative Distance? What?

  - by disappearedng

  I have a input file which are all floating point numbers to 4 decimal place. i.e. 13359 0.0000 0.0000 0.0001 0.0001 0.0002` 0.0003 0.0007 ... (the first is the id). My class uses the loadVectorsFromFile method which multiplies it by 10000 and then int() these numbers. On top of that, I also loop through each vector to ensure that there are no negative values inside. However, when I perform _hclustering, I am continually seeing the error, "Linkage Z contains negative values". I seriously think this is a bug because: I checked my values, the values are no where small enough or big enough to approach the limits of the floating point numbers and the formula that I used to derive the values in the file uses absolute value (my input is DEFINITELY right). Can someone enligten me as to why I am seeing this weird error? What is going on that is causing this negative distance error? ===== def loadVectorsFromFile(self, limit, loc, assertAllPositive=True, inflate=True): """Inflate to prevent "negative" distance, we use 4 decimal points, so *10000 """ vectors = {} self.winfo("Each vector is set to have %d limit in length" % limit) with open( loc ) as inf: for line in filter(None, inf.read().split('\n')): l = line.split('\t') if limit: scores = map(float, l[1:limit+1]) else: scores = map(float, l[1:]) if inflate: vectors[ l[0]] = map( lambda x: int(x*10000), scores) #int might save space else: vectors[ l[0]] = scores if assertAllPositive: #Assert that it has no negative value for dirID, l in vectors.iteritems(): if reduce(operator.or_, map( lambda x: x < 0, l)): self.werror( "Vector %s has negative values!" % dirID) return vectors def main( self, inputDir, outputDir, limit=0, inFname="data.vectors.all", mappingFname='all.id.features.group.intermediate'): """ Loads vector from a file and start clustering INPUT vectors is { featureID: tfidfVector (list), } """ IDFeatureDic = loadIdFeatureGroupDicFromIntermediate( pjoin(self.configDir, mappingFname)) if not os.path.exists(outputDir): os.makedirs(outputDir) vectors = self.loadVectorsFromFile( limit, pjoin( inputDir, inFname)) for threshold in map( lambda x:float(x)/30, range(20,30)): clusters = self._hclustering(threshold, vectors) if clusters: outputLoc = pjoin(outputDir, "threshold.%s.result" % str(threshold)) with open(outputLoc, 'w') as outf: for clusterNo, cluster in clusters.iteritems(): outf.write('%s\n' % str(clusterNo)) for featureID in cluster: feature, group = IDFeatureDic[featureID] outline = "%s\t%s\n" % (feature, group) outf.write(outline.encode('utf-8')) outf.write("\n") else: continue def _hclustering(self, threshold, vectors): """function which you should call to vary the threshold vectors: { featureID: [ tfidf scores, tfidf score, .. ] """ clusters = defaultdict(list) if len(vectors) > 1: try: results = hierarchy.fclusterdata( vectors.values(), threshold, metric='cosine') except ValueError, e: self.werror("_hclustering: %s" % str(e)) return False for i, featureID in enumerate( vectors.keys()):

  Read the article

• VB6 app not executing as scheduled task unless user is logged on

  - by Tedd Hansen

  Hi Would greatly apprechiate some help on this one! It may be a tricky one. :) Problem I have an VB6 application which is set up as scheduled task. It starts every time, but when executing CreateObject it fails if user is not logged on to computer. I am looking for information on what could cause this. Primary suspicion is that some Windows API fails. Key points Behaviour confirmed on Windows 2000, 2003, 2008 and Vista. The application executes as user X at scheduled time, executed by Windows Task Scheduler. It executes every time. Application does start! -- If user X is logged on via RDP it runs perfectly. (Note that user doesn't need to be connected, only logged on) -- If user X is not logged on to computer the application fails. Failure point Application fails when using CreateObject() to instansiate a DCOM object which is also part of the application. The DCOM objects declare .dll-references at startup (globally/on top of .bas-file) and run a small startup function. Failure must be during startup, possibly in one of the .dll-declarations. Thoughts After some Googling my initial suspicion was directed at MAPI. From what I could see MAPI required user to be logged on. The application has MAPI references. But even with all MAPI references removed it still does not work. What is the difference if an user is logged on? Registry mapping? Environment? explorer.exe is running. Isn't the user logged on when application executes as the user? What info would help? A definitive answer would be truly great. Any information regarding any VB6 feature/Windows API that could act differently depending on wether user is logged on or not would definitively help. Similar experiences may lead me in the right direction. Tips on debuggin this. Thanks! :)

  Read the article

• ASP.NET WebAPI Security 3: Extensible Authentication Framework

  - by Your DisplayName here!

  In my last post, I described the identity architecture of ASP.NET Web API. The short version was, that Web API (beta 1) does not really have an authentication system on its own, but inherits the client security context from its host. This is fine in many situations (e.g. AJAX style callbacks with an already established logon session). But there are many cases where you don’t use the containing web application for authentication, but need to do it yourself. Examples of that would be token based authentication and clients that don’t run in the context of the web application (e.g. desktop clients / mobile). Since Web API provides a nice extensibility model, it is easy to implement whatever security framework you want on top of it. My design goals were: Easy to use. Extensible. Claims-based. ..and of course, this should always behave the same, regardless of the hosting environment. In the rest of the post I am outlining some of the bits and pieces, So you know what you are dealing with, in case you want to try the code. At the very heart… is a so called message handler. This is a Web API extensibility point that gets to see (and modify if needed) all incoming and outgoing requests. Handlers run after the conversion from host to Web API, which means that handler code deals with HttpRequestMessage and HttpResponseMessage. See Pedro’s post for more information on the processing pipeline. This handler requires a configuration object for initialization. Currently this is very simple, it contains: Settings for the various authentication and credential types Settings for claims transformation Ability to block identity inheritance from host The most important part here is the credential type support, but I will come back to that later. The logic of the message handler is simple: Look at the incoming request. If the request contains an authorization header, try to authenticate the client. If this is successful, create a claims principal and populate the usual places. If not, return a 401 status code and set the Www-Authenticate header. Look at outgoing response, if the status code is 401, set the Www-Authenticate header. Credential type support Under the covers I use the WIF security token handler infrastructure to validate credentials and to turn security tokens into claims. The idea is simple: an authorization header consists of two pieces: the schema and the actual “token”. My configuration object allows to associate a security token handler with a scheme. This way you only need to implement support for a specific credential type, and map that to the incoming scheme value. The current version supports HTTP Basic Authentication as well as SAML and SWT tokens. (I needed to do some surgery on the standard security token handlers, since WIF does not directly support string-ified tokens. The next version of .NET will fix that, and the code should become simpler then). You can e.g. use this code to hook up a username/password handler to the Basic scheme (the default scheme name for Basic Authentication). config.Handler.AddBasicAuthenticationHandler( (username, password) => username == password); You simply have to provide a password validation function which could of course point back to your existing password library or e.g. membership. The following code maps a token handler for Simple Web Tokens (SWT) to the Bearer scheme (the currently favoured scheme name for OAuth2). You simply have to specify the issuer name, realm and shared signature key: config.Handler.AddSimpleWebTokenHandler(     "Bearer",     http://identity.thinktecture.com/trust,     Constants.Realm,     "Dc9Mpi3jaaaUpBQpa/4R7XtUsa3D/ALSjTVvK8IUZbg="); For certain integration scenarios it is very useful if your Web API can consume SAML tokens. This is also easily accomplishable. The following code uses the standard WIF API to configure the usual SAMLisms like issuer, audience, service certificate and certificate validation. Both SAML 1.1 and 2.0 are supported. var registry = new ConfigurationBasedIssuerNameRegistry(); registry.AddTrustedIssuer( "d1 c5 b1 25 97 d0 36 94 65 1c e2 64 fe 48 06 01 35 f7 bd db", "ADFS"); var adfsConfig = new SecurityTokenHandlerConfiguration(); adfsConfig.AudienceRestriction.AllowedAudienceUris.Add( new Uri(Constants.Realm)); adfsConfig.IssuerNameRegistry = registry; adfsConfig.CertificateValidator = X509CertificateValidator.None; // token decryption (read from configuration section) adfsConfig.ServiceTokenResolver = FederatedAuthentication.ServiceConfiguration.CreateAggregateTokenResolver(); config.Handler.AddSaml11SecurityTokenHandler("SAML", adfsConfig); Claims Transformation After successful authentication, if configured, the standard WIF ClaimsAuthenticationManager is called to run claims transformation and validation logic. This stage is used to transform the “technical” claims from the security token into application claims. You can either have a separate transformation logic, or share on e.g. with the containing web application. That’s just a matter of configuration. Adding the authentication handler to a Web API application In the spirit of Web API this is done in code, e.g. global.asax for web hosting: protected void Application_Start() {     AreaRegistration.RegisterAllAreas();     ConfigureApis(GlobalConfiguration.Configuration);     RegisterGlobalFilters(GlobalFilters.Filters);     RegisterRoutes(RouteTable.Routes);     BundleTable.Bundles.RegisterTemplateBundles(); } private void ConfigureApis(HttpConfiguration configuration) {     configuration.MessageHandlers.Add( new AuthenticationHandler(ConfigureAuthentication())); } private AuthenticationConfiguration ConfigureAuthentication() {     var config = new AuthenticationConfiguration     {         // sample claims transformation for consultants sample, comment out to see raw claims         ClaimsAuthenticationManager = new ApiClaimsTransformer(),         // value of the www-authenticate header, // if not set, the first scheme added to the handler collection is used         DefaultAuthenticationScheme = "Basic"     };     // add token handlers - see above     return config; } You can find the full source code and some samples here. In the next post I will describe some of the samples in the download, and then move on to authorization. HTH

  Read the article

• AVFoundation buffer comparison to a saved image

  - by user577552

  Hi, I am a long time reader, first time poster on StackOverflow, and must say it has been a great source of knowledge for me. I am trying to get to know the AVFoundation framework. What I want to do is save what the camera sees and then detect when something changes. Here is the part where I save the image to a UIImage : if (shouldSetBackgroundImage) { CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); // Create a bitmap graphics context with the sample buffer data CGContextRef context = CGBitmapContextCreate(rowBase, bufferWidth, bufferHeight, 8, bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst); // Create a Quartz image from the pixel data in the bitmap graphics context CGImageRef quartzImage = CGBitmapContextCreateImage(context); // Free up the context and color space CGContextRelease(context); CGColorSpaceRelease(colorSpace); // Create an image object from the Quartz image UIImage * image = [UIImage imageWithCGImage:quartzImage]; [self setBackgroundImage:image]; NSLog(@"reference image actually set"); // Release the Quartz image CGImageRelease(quartzImage); //Signal that the image has been saved shouldSetBackgroundImage = NO; } and here is the part where I check if there is any change in the image seen by the camera : else { CGImageRef cgImage = [backgroundImage CGImage]; CGDataProviderRef provider = CGImageGetDataProvider(cgImage); CFDataRef bitmapData = CGDataProviderCopyData(provider); char* data = CFDataGetBytePtr(bitmapData); if (data != NULL) { int64_t numDiffer = 0, pixelCount = 0; NSMutableArray * pointsMutable = [NSMutableArray array]; for( int row = 0; row < bufferHeight; row += 8 ) { for( int column = 0; column < bufferWidth; column += 8 ) { //we get one pixel from each source (buffer and saved image) unsigned char *pixel = rowBase + (row * bytesPerRow) + (column * BYTES_PER_PIXEL); unsigned char *referencePixel = data + (row * bytesPerRow) + (column * BYTES_PER_PIXEL); pixelCount++; if ( !match(pixel, referencePixel, matchThreshold) ) { numDiffer++; [pointsMutable addObject:[NSValue valueWithCGPoint:CGPointMake(SCREEN_WIDTH - (column/ (float) bufferHeight)* SCREEN_WIDTH - 4.0, (row/ (float) bufferWidth)* SCREEN_HEIGHT- 4.0)]]; } } } numberOfPixelsThatDiffer = numDiffer; points = [pointsMutable copy]; } For some reason, this doesn't work, meaning that the iPhone detects almost everything as being different from the saved image, even though I set a very low threshold for detection in the match function... Do you have any idea of what I am doing wrong?

  Read the article

• C# Begin/EndReceive - how do I read large data?

  - by ryeguy

  When reading data in chunks of say, 1024, how do I continue to read from a socket that receives a message bigger than 1024 bytes until there is no data left? Should I just use BeginReceive to read a packet's length prefix only, and then once that is retrieved, use Receive() (in the async thread) to read the rest of the packet? Or is there another way? edit: I thought Jon Skeet's link had the solution, but there is a bit of a speedbump with that code. The code I used is: public class StateObject { public Socket workSocket = null; public const int BUFFER_SIZE = 1024; public byte[] buffer = new byte[BUFFER_SIZE]; public StringBuilder sb = new StringBuilder(); } public static void Read_Callback(IAsyncResult ar) { StateObject so = (StateObject) ar.AsyncState; Socket s = so.workSocket; int read = s.EndReceive(ar); if (read > 0) { so.sb.Append(Encoding.ASCII.GetString(so.buffer, 0, read)); if (read == StateObject.BUFFER_SIZE) { s.BeginReceive(so.buffer, 0, StateObject.BUFFER_SIZE, 0, new AyncCallback(Async_Send_Receive.Read_Callback), so); return; } } if (so.sb.Length > 0) { //All of the data has been read, so displays it to the console string strContent; strContent = so.sb.ToString(); Console.WriteLine(String.Format("Read {0} byte from socket" + "data = {1} ", strContent.Length, strContent)); } s.Close(); } Now this corrected works fine most of the time, but it fails when the packet's size is a multiple of the buffer. The reason for this is if the buffer gets filled on a read it is assumed there is more data; but the same problem happens as before. A 2 byte buffer, for exmaple, gets filled twice on a 4 byte packet, and assumes there is more data. It then blocks because there is nothing left to read. The problem is that the receive function doesn't know when the end of the packet is. This got me thinking to two possible solutions: I could either have an end-of-packet delimiter or I could read the packet header to find the length and then receive exactly that amount (as I originally suggested). There's problems with each of these, though. I don't like the idea of using a delimiter, as a user could somehow work that into a packet in an input string from the app and screw it up. It also just seems kinda sloppy to me. The length header sounds ok, but I'm planning on using protocol buffers - I don't know the format of the data. Is there a length header? How many bytes is it? Would this be something I implement myself? Etc.. What should I do?

  Read the article

• Is it possible to change HANDLE that has been opened for synchronous I/O to be opened for asynchrono

  - by Martin Dobšík

  Dear all, Most of my daily programming work in Windows is nowadays around I/O operations of all kind (pipes, consoles, files, sockets, ...). I am well aware of different methods of reading and writing from/to different kinds of handles (Synchronous, asynchronous waiting for completion on events, waiting on file HANDLEs, I/O completion ports, and alertable I/O). We use many of those. For some of our applications it would be very useful to have only one way to treat all handles. I mean, the program may not know, what kind of handle it has received and we would like to use, let's say, I/O completion ports for all. So first I would ask: Let's assume I have a handle: HANDLE h; which has been received by my process for I/O from somewhere. Is there any easy and reliable way to find out what flags it has been created with? The main flag in question is FILE_FLAG_OVERLAPPED. The only way known to me so far, is to try to register such handle into I/O completion port (using CreateIoCompletionPort()). If that succeeds the handle has been created with FILE_FLAG_OVERLAPPED. But then only I/O completion port must be used, as the handle can not be unregistered from it without closing the HANDLE h itself. Providing there is an easy a way to determine presence of FILE_FLAG_OVERLAPPED, there would come my second question: Is there any way how to add such flag to already existing handle? That would make a handle that has been originally open for synchronous operations to be open for asynchronous. Would there be a way how to create opposite (remove the FILE_FLAG_OVERLAPPED to create synchronous handle from asynchronous)? I have not found any direct way after reading through MSDN and googling a lot. Would there be at least some trick that could do the same? Like re-creating the handle in same way using CreateFile() function or something similar? Something even partly documented or not documented at all? The main place where I would need this, is to determine the the way (or change the way) process should read/write from handles sent to it by third party applications. We can not control how third party products create their handles. Dear Windows gurus: help please! With regards Martin

  Read the article

• Android: database reading problem throws exception

  - by Vamsi

  Hi, i am having this problem with the android database. I adopted the DBAdapter file the NotepadAdv3 example from the google android page. DBAdapter.java public class DBAdapter { private static final String TAG = "DBAdapter"; private static final String DATABASE_NAME = "PasswordDb"; private static final String DATABASE_TABLE = "myuserdata"; private static final String DATABASE_USERKEY = "myuserkey"; private static final int DATABASE_VERSION = 2; public static final String KEY_USERKEY = "userkey"; public static final String KEY_TITLE = "title"; public static final String KEY_DATA = "data"; public static final String KEY_ROWID = "_id"; private final Context mContext; private DatabaseHelper mDbHelper; private SQLiteDatabase mDb; private static final String DB_CREATE_KEY = "create table " + DATABASE_USERKEY + " (" + "userkey text not null" +");"; private static final String DB_CREATE_DATA = "create table " + DATABASE_TABLE + " (" + "_id integer primary key autoincrement, " + "title text not null" + "data text" +");"; private static class DatabaseHelper extends SQLiteOpenHelper { DatabaseHelper(Context context) { super(context, DATABASE_NAME, null, DATABASE_VERSION); } @Override public void onCreate(SQLiteDatabase db) { db.execSQL(DB_CREATE_KEY); db.execSQL(DB_CREATE_DATA); } @Override public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) { Log.w(TAG, "Upgrading database from version " + oldVersion + " to " + newVersion + ", which will destroy all old data"); db.execSQL("DROP TABLE IF EXISTS myuserkey"); db.execSQL("DROP TABLE IF EXISTS myuserdata"); onCreate(db); } } public DBAdapter(Context ctx) { this.mContext = ctx; } public DBAdapter Open() throws SQLException{ try { mDbHelper = new DatabaseHelper(mContext); } catch(Exception e){ Log.e(TAG, e.toString()); } mDb = mDbHelper.getWritableDatabase(); return this; } public void close(){ mDbHelper.close(); } public Long storeKey(String userKey){ ContentValues initialValues = new ContentValues(); initialValues.put(KEY_USERKEY, userKey); try { mDb.delete(DATABASE_USERKEY, "1=1", null); } catch(Exception e) { Log.e(TAG, e.toString()); } return mDb.insert(DATABASE_USERKEY, null, initialValues); } public String retrieveKey() { final Cursor c; try { c = mDb.query(DATABASE_USERKEY, new String[] { KEY_USERKEY}, null, null, null, null, null); }catch(Exception e){ Log.e(TAG, e.toString()); return ""; } if(c.moveToFirst()){ return c.getString(0); } else{ Log.d(TAG, "UserKey Empty"); } return ""; } //not including any function related to "myuserdata" table } Class1.java { mUserKey = mDbHelper.retrieveKey(); mDbHelper.storeKey(Key); } the error that i am receiving is from Log.e(TAG, e.toString()) in the methods retrieveKey() and storeKey() "no such table: myuserkey: , while compiling: SELECT userkey FROM myuserkey"

  Read the article

• IoC/DI in the face of winforms and other generated code

  - by Kaleb Pederson

  When using dependency injection (DI) and inversion of control (IoC) objects will typically have a constructor that accepts the set of dependencies required for the object to function properly. For example, if I have a form that requires a service to populate a combo box you might see something like this: // my files public interface IDataService { IList<MyData> GetData(); } public interface IComboDataService { IList<MyComboData> GetComboData(); } public partial class PopulatedForm : BaseForm { private IDataService service; public PopulatedForm(IDataService service) { //... InitializeComponent(); } } This works fine at the top level, I just use my IoC container to resolve the dependencies: var form = ioc.Resolve<PopulatedForm>(); But in the face of generated code, this gets harder. In winforms a second file composing the rest of the partial class is generated. This file references other components, such as custom controls, and uses no-args constructors to create such controls: // generated file: PopulatedForm.Designer.cs public partial class PopulatedForm { private void InitializeComponent() { this.customComboBox = new UserCreatedComboBox(); // customComboBox has an IComboDataService dependency } } Since this is generated code, I can't pass in the dependencies and there's no easy way to have my IoC container automatically inject all the dependencies. One solution is to pass in the dependencies of each child component to PopulatedForm even though it may not need them directly, such as with the IComboDataService required by the UserCreatedComboBox. I then have the responsibility to make sure that the dependencies are provided through various properties or setter methods. Then, my PopulatedForm constructor might look as follows: public PopulatedForm(IDataService service, IComboDataService comboDataService) { this.service = service; InitializeComponent(); this.customComboBox.ComboDataService = comboDataService; } Another possible solution is to have the no-args constructor to do the necessary resolution: public class UserCreatedComboBox { private IComboDataService comboDataService; public UserCreatedComboBox() { if (!DesignMode && IoC.Instance != null) { comboDataService = Ioc.Instance.Resolve<IComboDataService>(); } } } Neither solution is particularly good. What patterns and alternatives are available to more capably handle dependency-injection in the face of generated code? I'd love to see both general solutions, such as patterns, and ones specific to C#, Winforms, and Autofac.

  Read the article

• Uploading on Youtube via HTTP Post

  - by sajid.nizami

  I am following the steps provided on this link [http://code.google.com/apis/youtube/2.0/developers_guide_dotnet.html#Browser_based_Upload][1] Whenever I try to upload anything using this method, I get a HTTP 400 error saying that the next_url is not provided. Code is pretty simple and is a copy of Google's own code. <%@ Page Language="C#" AutoEventWireup="true" CodeFile="BrowserUpload.aspx.cs" Inherits="BrowserUpload" %> <%@ Import Namespace="Google.YouTube" %> <%@ Import Namespace="Google.GData.Extensions.MediaRss" %> <%@ Import Namespace="Google.GData" %> <%@ Import Namespace="Google.GData.YouTube" %> <%@ Import Namespace="Google.GData.Client" %> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head runat="server"> <title></title> <script type="text/javascript"> function checkForFile() { if (document.getElementById('file').value) { return true; } document.getElementById('errMsg').style.display = ''; return false; } </script> </head> <body> <% YouTubeRequestSettings settings = new YouTubeRequestSettings("Danat", "API-KEY", "loginid", "password" ); YouTubeRequest request = new YouTubeRequest(settings); Video newVideo = new Video(); newVideo.Title = "My Test Movie"; newVideo.Tags.Add(new MediaCategory("Autos", YouTubeNameTable.CategorySchema)); newVideo.Keywords = "cars, funny"; newVideo.Description = "My description"; newVideo.YouTubeEntry.Private = false; newVideo.Tags.Add(new MediaCategory("mydevtag, anotherdevtag", YouTubeNameTable.DeveloperTagSchema)); FormUploadToken token = request.CreateFormUploadToken(newVideo); %> <form action="<%= token.Url %>?next_url=<%= Server.UrlEncode("http://www.danatev.com") %>" name="PostToYoutube" method="post" enctype="multipart/form-data" onsubmit="return checkForFile();" > <input id="file" type="file" name="file" /> <div id="errMsg" style="display: none; color: red"> You need to specify a file. </div> <input type="hidden" name="token" value="<%= token.Token %>" /> <input type="submit" value="go" /> </form>

  Read the article

• PHP fopen returning null on files that work fine with include and get_file_contents

  - by brad allred

  Hi, I have XAMPP installed on a windows 2000 server. everything is working great except the PHP fopen function. I can neither create nor open files with it. the strange thing is that i can include/require/file_get_contents/other file related functions; also fopen does not generate any errors or notices it just returns NULL. I have gone as far as to grant full control of the file and all enclosing folders to everybody but i still get NULL instead of a file pointer. I have tried this on php 5.2.9, 5.2.13, and 5.3.1 with the same effect. i have gone through the php.ini file looking for something that is breaking it; i have even tried deleting and using the basic ini file from a linux box where fopen is working and still nothing. I know i have to restart apache after changing my ini and all that and have been (I have even restarted the server) so thats not it. I am at this poing assuming it is an apache configuration issue somehow, tomorrow im going to run a test through php-cli to make sure. I really don't want to bruise my head anymore over this can some apache/php wizard come to my aid? Hi guys, thanks for the responses. you are right is is not any config problem. the problem has to be with one of my dlls or one of my included files. I just tried the same code that isn't working in a new file without any include and i disabled my custom libraries and it worked. for the record here is what I was doing that wasn't working: $test_file = 'c:\\test.csv';//everybody has full control. is very large. if(file_exists($test_file) && is_readable($test_file)){ $fp = fopen($test_file, 'r'); echo var_export($fp, true);//outputs NULL. on my linux box this is a number. if($fp !== false){ //do the work fread($fp, 10);//throws the error that $fp is not a valid file handle } } something that i am including must be breaking fopen somehow. works as expected in new file with no includes.

  Read the article

• Why is numpy's einsum faster than numpy's built in functions?

  - by Ophion

  Lets start with three arrays of dtype=np.double. Timings are performed on a intel CPU using numpy 1.7.1 compiled with icc and linked to intel's mkl. A AMD cpu with numpy 1.6.1 compiled with gcc without mkl was also used to verify the timings. Please note the timings scale nearly linearly with system size and are not due to the small overhead incurred in the numpy functions if statements these difference will show up in microseconds not milliseconds: arr_1D=np.arange(500,dtype=np.double) large_arr_1D=np.arange(100000,dtype=np.double) arr_2D=np.arange(500**2,dtype=np.double).reshape(500,500) arr_3D=np.arange(500**3,dtype=np.double).reshape(500,500,500) First lets look at the np.sum function: np.all(np.sum(arr_3D)==np.einsum('ijk->',arr_3D)) True %timeit np.sum(arr_3D) 10 loops, best of 3: 142 ms per loop %timeit np.einsum('ijk->', arr_3D) 10 loops, best of 3: 70.2 ms per loop Powers: np.allclose(arr_3D*arr_3D*arr_3D,np.einsum('ijk,ijk,ijk->ijk',arr_3D,arr_3D,arr_3D)) True %timeit arr_3D*arr_3D*arr_3D 1 loops, best of 3: 1.32 s per loop %timeit np.einsum('ijk,ijk,ijk->ijk', arr_3D, arr_3D, arr_3D) 1 loops, best of 3: 694 ms per loop Outer product: np.all(np.outer(arr_1D,arr_1D)==np.einsum('i,k->ik',arr_1D,arr_1D)) True %timeit np.outer(arr_1D, arr_1D) 1000 loops, best of 3: 411 us per loop %timeit np.einsum('i,k->ik', arr_1D, arr_1D) 1000 loops, best of 3: 245 us per loop All of the above are twice as fast with np.einsum. These should be apples to apples comparisons as everything is specifically of dtype=np.double. I would expect the speed up in an operation like this: np.allclose(np.sum(arr_2D*arr_3D),np.einsum('ij,oij->',arr_2D,arr_3D)) True %timeit np.sum(arr_2D*arr_3D) 1 loops, best of 3: 813 ms per loop %timeit np.einsum('ij,oij->', arr_2D, arr_3D) 10 loops, best of 3: 85.1 ms per loop Einsum seems to be at least twice as fast for np.inner, np.outer, np.kron, and np.sum regardless of axes selection. The primary exception being np.dot as it calls DGEMM from a BLAS library. So why is np.einsum faster that other numpy functions that are equivalent? The DGEMM case for completeness: np.allclose(np.dot(arr_2D,arr_2D),np.einsum('ij,jk',arr_2D,arr_2D)) True %timeit np.einsum('ij,jk',arr_2D,arr_2D) 10 loops, best of 3: 56.1 ms per loop %timeit np.dot(arr_2D,arr_2D) 100 loops, best of 3: 5.17 ms per loop The leading theory is from @sebergs comment that np.einsum can make use of SSE2, but numpy's ufuncs will not until numpy 1.8 (see the change log). I believe this is the correct answer, but have not been able to confirm it. Some limited proof can be found by changing the dtype of input array and observing speed difference and the fact that not everyone observes the same trends in timings.

  Read the article

• Is it possible to programatically catch JavaScript SyntaxErrors?

  - by Matty

  I don't think that this is doable, but wanted to throw it out to the community to confirm my suspicions. Let's say you're writing a program in another language like Python, PHP, or ASP. This program is intended to build another program written in JavaScript. However, the first program is unfortunately not immune to errors. So, occasionally, the program which builds the JavaScript program does some funky stuff and outputs a syntax error in the JavaScript source. Now some user goes and loads the program and it essentially halts, because the web browser running it can't properly parse the JavaScript. This user is probably not going to be happy. I wouldn't be. This brings me to my question. Is it possible to write an error handler that would catch these kind of syntax problems allowing the application to fail gracefully? Here's an example: <html> <head> <script type="text/javascript" charset="utf-8"> window.onerror = errorHandler; function errorHandler(a,b,c) { alert('horray! No wait, Booo!'); } vara jfs; </script> </head> <body> Can this be done? </body> </html> or <html> <head> <script type="text/javascript" charset="utf-8"> try { vara jfs; } catch (e) { alert('horray! No wait, Booo!'); } </script> </head> <body> Can this be done? </body> </html>

  Read the article

• SQL Server Table Polling by Multiple Subscribers

  - by Daniel Hester

  Background Designing Stored Procedures that are safe for multiple subscribers (to call simultaneously) can be challenging.  For example let’s say that you want multiple worker processes to poll a shared work queue that’s encapsulated as a SQL Table. This is a common scenario and through experience you’ll find that you want to use Table Hints to prevent unwanted locking when performing simultaneous queries on the same table. There are three table hints to consider: NOLOCK, READPAST and UPDLOCK. Both NOLOCK and READPAST table hints allow you to SELECT from a table without placing a LOCK on that table. However, SELECTs with the READPAST hint will ignore any records that are locked due to being updated/inserted (or otherwise “dirty”), whereas a SELECT with NOLOCK ignores all locks including dirty reads. For the initial update of the flag (that marks the record as available for subscription) I don’t use the NOLOCK Table Hint because I want to be sensitive to the “active” records in the table and I want to exclude them.  I use an Update Lock (UPDLOCK) in conjunction with a WHERE clause that uses a sub-select with a READPAST Table Hint in order to explicitly lock the records I’m updating (UPDLOCK) but not place a lock on the table when selecting the records that I’m going to update (READPAST). UPDATES should be allowed to lock the rows affected because we’re probably changing a flag on a record so that it is not included in a SELECT from another subscriber. On the UPDATE statement we should explicitly use the UPDLOCK to guard against lock escalation. A SELECT to check for the next record(s) to process can result in a shared read lock being held by more than one subscriber polling the shared work queue (SQL table). It is expected that more than one worker process (or server) might try to process the same new record(s) at the same time. When each process then tries to obtain the update lock, none of them can because another process has a shared read lock in place. Thus without the UPDLOCK hint the result would be a lock escalation deadlock; however with the UPDLOCK hint this condition is mitigated against. Note that using the READPAST table hint requires that you also set the ISOLATION LEVEL of the transaction to be READ COMMITTED (rather than the default of SERIALIZABLE). Guidance In the Stored Procedure that returns records to the multiple subscribers: Perform the UPDATE first. Change the flag that makes the record available to subscribers.  Additionally, you may want to update a LastUpdated datetime field in order to be able to check for records that “got stuck” in an intermediate state or for other auditing purposes. In the UPDATE statement use the (UPDLOCK) Table Hint on the UPDATE statement to prevent lock escalation. In the UPDATE statement also use a WHERE Clause that uses a sub-select with a (READPAST) Table Hint to select the records that you’re going to update. In the UPDATE statement use the OUTPUT clause in conjunction with a Temporary Table to isolate the record(s) that you’ve just updated and intend to return to the subscriber. This is the fastest way to update the record(s) and to get the records’ identifiers within the same operation. Finally do a set-based SELECT on the main Table (using the Temporary Table to identify the records in the set) with either a READPAST or NOLOCK table hint.  Use NOLOCK if there are other processes (besides the multiple subscribers) that might be changing the data that you want to return to the multiple subscribers; or use READPAST if you're sure there are no other processes (besides the multiple subscribers) that might be updating column data in the table for other purposes (e.g. changes to a person’s last name).  NOLOCK is generally the better fit in this part of the scenario. See the following as an example: CREATE PROCEDURE [dbo].[usp_NewCustomersSelect] AS BEGIN -- OVERRIDE THE DEFAULT ISOLATION LEVEL SET TRANSACTION ISOLATION LEVEL READ COMMITTED -- SET NOCOUNT ON SET NOCOUNT ON -- DECLARE TEMP TABLE -- Note that this example uses CustomerId as an identifier; -- you could just use the Identity column Id if that’s all you need. DECLARE @CustomersTempTable TABLE ( CustomerId NVARCHAR(255) ) -- PERFORM UPDATE FIRST -- [Customers] is the name of the table -- [Id] is the Identity Column on the table -- [CustomerId] is the business document key used to identify the -- record globally, i.e. in other systems or across SQL tables -- [Status] is INT or BIT field (if the status is a binary state) -- [LastUpdated] is a datetime field used to record the time of the -- last update UPDATE [Customers] WITH (UPDLOCK) SET [Status] = 1, [LastUpdated] = GETDATE() OUTPUT [INSERTED].[CustomerId] INTO @CustomersTempTable WHERE ([Id] = (SELECT TOP 100 [Id] FROM [Customers] WITH (READPAST) WHERE ([Status] = 0) ORDER BY [Id] ASC)) -- PERFORM SELECT FROM ENTITY TABLE SELECT [C].[CustomerId], [C].[FirstName], [C].[LastName], [C].[Address1], [C].[Address2], [C].[City], [C].[State], [C].[Zip], [C].[ShippingMethod], [C].[Id] FROM [Customers] AS [C] WITH (NOLOCK), @CustomersTempTable AS [TEMP] WHERE ([C].[CustomerId] = [TEMP].[CustomerId]) END In a system that has been designed to have multiple status values for records that need to be processed in the Work Queue it is necessary to have a “Watch Dog” process by which “stale” records in intermediate states (such as “In Progress”) are detected, i.e. a [Status] of 0 = New or Unprocessed; a [Status] of 1 = In Progress; a [Status] of 2 = Processed; etc.. Thus, if you have a business rule that states that the application should only process new records if all of the old records have been processed successfully (or marked as an error), then it will be necessary to build a monitoring process to detect stalled or stale records in the Work Queue, hence the use of the LastUpdated column in the example above. The Status field along with the LastUpdated field can be used as the criteria to detect stalled / stale records. It is possible to put this watchdog logic into the stored procedure above, but I would recommend making it a separate monitoring function. In writing the stored procedure that checks for stale records I would recommend using the same kind of lock semantics as suggested above. The example below looks for records that have been in the “In Progress” state ([Status] = 1) for greater than 60 seconds: CREATE PROCEDURE [dbo].[usp_NewCustomersWatchDog] AS BEGIN -- TO OVERRIDE THE DEFAULT ISOLATION LEVEL SET TRANSACTION ISOLATION LEVEL READ COMMITTED -- SET NOCOUNT ON SET NOCOUNT ON DECLARE @MaxWait int; SET @MaxWait = 60 IF EXISTS (SELECT 1 FROM [dbo].[Customers] WITH (READPAST) WHERE ([Status] = 1) AND (DATEDIFF(s, [LastUpdated], GETDATE()) > @MaxWait)) BEGIN SELECT 1 AS [IsWatchDogError] END ELSE BEGIN SELECT 0 AS [IsWatchDogError] END END Downloads The zip file below contains two SQL scripts: one to create a sample database with the above stored procedures and one to populate the sample database with 10,000 sample records.  I am very grateful to Red-Gate software for their excellent SQL Data Generator tool which enabled me to create these sample records in no time at all. References http://msdn.microsoft.com/en-us/library/ms187373.aspx http://www.techrepublic.com/article/using-nolock-and-readpast-table-hints-in-sql-server/6185492 http://geekswithblogs.net/gwiele/archive/2004/11/25/15974.aspx http://grounding.co.za/blogs/romiko/archive/2009/03/09/biztalk-sql-receive-location-deadlocks-dirty-reads-and-isolation-levels.aspx

  Read the article

• understanding logic of dijit css and styles

  - by Tom

  Hi, I am trying to use dijit.InlineEditBox. I have put the following code in my HTML, using the example in the dojo docs: <script type="text/javascript"> dojo.require("dijit.InlineEditBox"); dojo.require("dojo.parser"); dojo.require("dijit.form.TextBox"); function editableHeaderOnChange(id, arg){ alert("details changed with id " + id + " and arguments "+arg); } </script> ... <span id="myText" dojoType="dijit.InlineEditBox" onChange="editableHeaderOnChange(this.id,arguments[0])" autoSave="true" title="My Text">click to edit me</span> I am using tundra theme. It works, however it doesn't look so good. The widget has its own style, which doesn't fit my CSS. I used firebug to locate the source of the problem. The widget creates many nested div/span elements, each has it's own style (element style in firebug): <span id="dijit__InlineEditor_0" class="dijitReset dijitInline" style="margin: 0px; position: absolute; visibility: hidden; display: block; opacity: 0;" ...> <input type="text" autocomplete="off" class="dijit dijitReset dijitLeft dijitTextBox" id="dijit_form_TextBox_0" style="line-height: 20px; font-weight: 400; font-family: Trebuchet MS,Helvetica,Arial,Verdana; font-size: 14.5167px; font-style: normal; width: 100%;"> ...> </span></span> (showing only the relevant parts...) to get the visual that I want, which will not break to a newline, I need to change the width of dijit_form_TextBox_0** to 50%, and the positioning of dijit__InlineEditor_0 to display: inline**; or to change the positioning of everything (most of my layout is floated, so position: absolute doesn't fit) I cannot address those span elements in my css to change the properties, because the element.style has priority, of course. I don't understand the logic in this system... why is dijit generating the style directly inside the element? how can I change these properties? Thanks Tom

  Read the article

• How to write this Linq SQL as a Dynamic Query (using strings)?

  - by Dr. Zim

  Skip to the "specific question" as needed. Some background: The scenario: I have a set of products with a "drill down" filter (Query Object) populated with DDLs. Each progressive DDL selection will further limit the product list as well as what options are left for the DDLs. For example, selecting a hammer out of tools limits the Product Sizes to only show hammer sizes. Current setup: I created a query object, sent it to a repository, and fed each option to a SQL "table valued function" where null values represent "get all products". I consider this a good effort, but far from DDD acceptable. I want to avoid any "programming" in SQL, hopefully doing everything with a repository. Comments on this topic would be appreciated. Specific question: How would I rewrite this query as a Dynamic Query? A link to something like 101 Linq Examples would be fantastic, but with a Dynamic Query scope. I really want to pass to this method the field in quotes "" for which I want a list of options and how many products have that option. from p in db.Products group p by p.ProductSize into g select new Category { PropertyType = g.Key, Count = g.Count() } Each DDL option will have "The selection (21)" where the (21) is the quantity of products that have that attribute. Upon selecting an option, all other remaining DDLs will update with the remaining options and counts. Edit: Additional notes: .OrderBy("it.City") // "it" refers to the entire record .GroupBy("City", "new(City)") // This produces a unique list of City .Select("it.Count()") //This gives a list of counts... getting closer .Select("key") // Selects a list of unique City .Select("new (key, count() as string)") // +1 to me LOL. key is a row of group .GroupBy("new (City, Manufacturer)", "City") // New = list of fields to group by .GroupBy("City", "new (Manufacturer, Size)") // Second parameter is a projection Product .Where("ProductType == @0", "Maps") .GroupBy("new(City)", "new ( null as string)")// Projection not available later? .Select("new (key.City, it.count() as string)")// GroupBy new makes key an object Product .Where("ProductType == @0", "Maps") .GroupBy("new(City)", "new ( null as string)")// Projection not available later? .Select("new (key.City, it as object)")// the it object is the result of GroupBy var a = Product .Where("ProductType == @0", "Maps") .GroupBy("@0", "it", "City") // This fails to group Product at all .Select("new ( Key, it as Product )"); // "it" is property cast though What I have learned so far is LinqPad is fantastic, but still looking for an answer. Eventually, completely random research like this will prevail I guess. LOL. Edit:

  Read the article

• I'm cloning a table row that contains an input that's being set to jQuery TimeEntry that errors when

  - by Kendall Crouch

  I'm adding a TimeEntry to a page where the user can add a row (clone) to a table. The row that is cloned is hidden (display:none). The user clicks a button and javascript is run to clone the row which renames all of the fields and then appends the new row to the table. <tr id="blankRowShift"> <td> <input type="text" id="timeStart" name="timeStart" /> </td> <td> <input type="text" id="timeEnd" name="timeEnd" /> </td> <td> <select id="userLevel"> <option value="0">Please Select One</option> <option value="2">Admin</option> <option value="1">Employee</option> <option value="3">Scheduler</option> </select> </td> </tr> var r = $("#tbl #blankRowShift").clone().removeAttr("id"); $("#timeStart", r).attr("name", "timeStart" + nn).attr("id", "timeStart" + nn); $("#timeEnd", r).attr("name", "timeEnd" + nn).attr("id", "timeEnd" + nn); $("#userLevel option:nth(0)", r).attr("selected", "selected"); $("#userLevel", r).attr("name", "userLevel" + nn).attr("id", "userLevel" + nn).attr("value", 0); $("#tbl").append(r); $("#timeStart" + nn).timeEntry({ show24Hours: false, showSeconds: false, timeSteps: [1, 15, 0], spinnerImage: 'includes/js/spinnerOrange.png', beforeShow: customRangeStart }); $("#timeStart" + nn).timeEntry('setTime', new Date()); $("#timeEnd" + nn).timeEntry({ show24Hours: false, showSeconds: false, timeSteps: [1, 15, 0], spinnerImage: 'includes/js/spinnerOrange.png', beforeShow: customRangeEnd }); $("#timeEnd" + nn).timeEntry('setTime', new Date()); The spinner works just fine and the times can be changed. Then when submitting the form, I validate the time. The getTime errors in jQuery with the message "elem is undefined var id = elem[ expando ];". I've placed the statement 'console.dir(input)' in the _getTimeTimeEntry: function and it returns nothing for the cloned fields. el = $("#timeStart" + i); if (el.timeEntry("getTime") == null) {

  Read the article

• Convert List of one type to Array of another type using Dozer

  - by aheu

  I'm wondering how to convert a List of one type to an array of another type in Java using Dozer. The two types have all the same property names/types. For example, consider these two classes. public class A{ private String test = null; public String getTest(){ return this.test } public void setTest(String test){ this.test = test; } } public class B{ private String test = null; public String getTest(){ return this.test } public void setTest(String test){ this.test = test; } } I've tried this with no luck. List<A> listOfA = getListofAObjects(); Mapper mapper = DozerBeanMapperSingletonWrapper.getInstance(); B[] bs = mapper.map(listOfA, B[].class); I've also tried using the CollectionUtils class. CollectionUtils.convertListToArray(listOfA, B.class) Neither are working for me, can anyone tell me what I am doing wrong? The mapper.map function works fine if I create two wrapper classes, one containing a List and the other a b[]. See below: public class C{ private List<A> items = null; public List<A> getItems(){ return this.items; } public void setItems(List<A> items){ this.items = items; } } public class D{ private B[] items = null; public B[] getItems(){ return this.items; } public void setItems(B[] items){ this.items = items; } } This works oddly enough... List<A> listOfA = getListofAObjects(); C c = new C(); c.setItems(listOfA); Mapper mapper = DozerBeanMapperSingletonWrapper.getInstance(); D d = mapper.map(listOfA, D.class); B[] bs = d.getItems(); How do I do what I want to do without using the wrapper classes (C & D)? There has got to be an easier way... Thanks!

  Read the article

• Rush Hour - Solving the game

  - by Rubys

  Rush Hour if you're not familiar with it, the game consists of a collection of cars of varying sizes, set either horizontally or vertically, on a NxM grid that has a single exit. Each car can move forward/backward in the directions it's set in, as long as another car is not blocking it. You can never change the direction of a car. There is one special car, usually it's the red one. It's set in the same row that the exit is in, and the objective of the game is to find a series of moves (a move - moving a car N steps back or forward) that will allow the red car to drive out of the maze. I've been trying to think how to solve this problem computationally, and I can really not think of any good solution. I came up with a few: Backtracking. This is pretty simple - Recursion and some more recursion until you find the answer. However, each car can be moved a few different ways, and in each game state a few cars can be moved, and the resulting game tree will be HUGE. Some sort of constraint algorithm that will take into account what needs to be moved, and work recursively somehow. This is a very rough idea, but it is an idea. Graphs? Model the game states as a graph and apply some sort of variation on a coloring algorithm, to resolve dependencies? Again, this is a very rough idea. A friend suggested genetic algorithms. This is sort of possible but not easily. I can't think of a good way to make an evaluation function, and without that we've got nothing. So the question is - How to create a program that takes a grid and the vehicle layout, and outputs a series of steps needed to get the red car out? Sub-issues: Finding some solution. Finding an optimal solution (minimal number of moves) Evaluating how good a current state is Example: How can you move the cars in this setting, so that the red car can "exit" the maze through the exit on the right?

  Read the article

< Previous Page | 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451  | Next Page >