Search Results

Search found 7638 results on 306 pages for 'binary tree'.

Page 102/306 | < Previous Page | 98 99 100 101 102 103 104 105 106 107 108 109  | Next Page >

• What's up with OCFS2?

  - by wcoekaer

  On Linux there are many filesystem choices and even from Oracle we provide a number of filesystems, all with their own advantages and use cases. Customers often confuse ACFS with OCFS or OCFS2 which then causes assumptions to be made such as one replacing the other etc... I thought it would be good to write up a summary of how OCFS2 got to where it is, what we're up to still, how it is different from other options and how this really is a cool native Linux cluster filesystem that we worked on for many years and is still widely used. Work on a cluster filesystem at Oracle started many years ago, in the early 2000's when the Oracle Database Cluster development team wrote a cluster filesystem for Windows that was primarily focused on providing an alternative to raw disk devices and help customers with the deployment of Oracle Real Application Cluster (RAC). Oracle RAC is a cluster technology that lets us make a cluster of Oracle Database servers look like one big database. The RDBMS runs on many nodes and they all work on the same data. It's a Shared Disk database design. There are many advantages doing this but I will not go into detail as that is not the purpose of my write up. Suffice it to say that Oracle RAC expects all the database data to be visible in a consistent, coherent way, across all the nodes in the cluster. To do that, there were/are a few options : 1) use raw disk devices that are shared, through SCSI, FC, or iSCSI 2) use a network filesystem (NFS) 3) use a cluster filesystem(CFS) which basically gives you a filesystem that's coherent across all nodes using shared disks. It is sort of (but not quite) combining option 1 and 2 except that you don't do network access to the files, the files are effectively locally visible as if it was a local filesystem. So OCFS (Oracle Cluster FileSystem) on Windows was born. Since Linux was becoming a very important and popular platform, we decided that we would also make this available on Linux and thus the porting of OCFS/Windows started. The first version of OCFS was really primarily focused on replacing the use of Raw devices with a simple filesystem that lets you create files and provide direct IO to these files to get basically native raw disk performance. The filesystem was not designed to be fully POSIX compliant and it did not have any where near good/decent performance for regular file create/delete/access operations. Cache coherency was easy since it was basically always direct IO down to the disk device and this ensured that any time one issues a write() command it would go directly down to the disk, and not return until the write() was completed. Same for read() any sort of read from a datafile would be a read() operation that went all the way to disk and return. We did not cache any data when it came down to Oracle data files. So while OCFS worked well for that, since it did not have much of a normal filesystem feel, it was not something that could be submitted to the kernel mail list for inclusion into Linux as another native linux filesystem (setting aside the Windows porting code ...) it did its job well, it was very easy to configure, node membership was simple, locking was disk based (so very slow but it existed), you could create regular files and do regular filesystem operations to a certain extend but anything that was not database data file related was just not very useful in general. Logfiles ok, standard filesystem use, not so much. Up to this point, all the work was done, at Oracle, by Oracle developers. Once OCFS (1) was out for a while and there was a lot of use in the database RAC world, many customers wanted to do more and were asking for features that you'd expect in a normal native filesystem, a real "general purposes cluster filesystem". So the team sat down and basically started from scratch to implement what's now known as OCFS2 (Oracle Cluster FileSystem release 2). Some basic criteria were : Design it with a real Distributed Lock Manager and use the network for lock negotiation instead of the disk Make it a Linux native filesystem instead of a native shim layer and a portable core Support standard Posix compliancy and be fully cache coherent with all operations Support all the filesystem features Linux offers (ACL, extended Attributes, quotas, sparse files,...) Be modern, support large files, 32/64bit, journaling, data ordered journaling, endian neutral, we can mount on both endian /cross architecture,.. Needless to say, this was a huge development effort that took many years to complete. A few big milestones happened along the way... OCFS2 was development in the open, we did not have a private tree that we worked on without external code review from the Linux Filesystem maintainers, great folks like Christopher Hellwig reviewed the code regularly to make sure we were not doing anything out of line, we submitted the code for review on lkml a number of times to see if we were getting close for it to be included into the mainline kernel. Using this development model is standard practice for anyone that wants to write code that goes into the kernel and having any chance of doing so without a complete rewrite or.. shall I say flamefest when submitted. It saved us a tremendous amount of time by not having to re-fit code for it to be in a Linus acceptable state. Some other filesystems that were trying to get into the kernel that didn't follow an open development model had a lot harder time and a lot harsher criticism. March 2006, when Linus released 2.6.16, OCFS2 officially became part of the mainline kernel, it was accepted a little earlier in the release candidates but in 2.6.16. OCFS2 became officially part of the mainline Linux kernel tree as one of the many filesystems. It was the first cluster filesystem to make it into the kernel tree. Our hope was that it would then end up getting picked up by the distribution vendors to make it easy for everyone to have access to a CFS. Today the source code for OCFS2 is approximately 85000 lines of code. We made OCFS2 production with full support for customers that ran Oracle database on Linux, no extra or separate support contract needed. OCFS2 1.0.0 started being built for RHEL4 for x86, x86-64, ppc, s390x and ia64. For RHEL5 starting with OCFS2 1.2. SuSE was very interested in high availability and clustering and decided to build and include OCFS2 with SLES9 for their customers and was, next to Oracle, the main contributor to the filesystem for both new features and bug fixes. Source code was always available even prior to inclusion into mainline and as of 2.6.16, source code was just part of a Linux kernel download from kernel.org, which it still is, today. So the latest OCFS2 code is always the upstream mainline Linux kernel. OCFS2 is the cluster filesystem used in Oracle VM 2 and Oracle VM 3 as the virtual disk repository filesystem. Since the filesystem is in the Linux kernel it's released under the GPL v2 The release model has always been that new feature development happened in the mainline kernel and we then built consistent, well tested, snapshots that had versions, 1.2, 1.4, 1.6, 1.8. But these releases were effectively just snapshots in time that were tested for stability and release quality. OCFS2 is very easy to use, there's a simple text file that contains the node information (hostname, node number, cluster name) and a file that contains the cluster heartbeat timeouts. It is very small, and very efficient. As Sunil Mushran wrote in the manual : OCFS2 is an efficient, easily configured, quickly installed, fully integrated and compatible, feature-rich, architecture and endian neutral, cache coherent, ordered data journaling, POSIX-compliant, shared disk cluster file system. Here is a list of some of the important features that are included : Variable Block and Cluster sizes Supports block sizes ranging from 512 bytes to 4 KB and cluster sizes ranging from 4 KB to 1 MB (increments in power of 2). Extent-based Allocations Tracks the allocated space in ranges of clusters making it especially efficient for storing very large files. Optimized Allocations Supports sparse files, inline-data, unwritten extents, hole punching and allocation reservation for higher performance and efficient storage. File Cloning/snapshots REFLINK is a feature which introduces copy-on-write clones of files in a cluster coherent way. Indexed Directories Allows efficient access to millions of objects in a directory. Metadata Checksums Detects silent corruption in inodes and directories. Extended Attributes Supports attaching an unlimited number of name:value pairs to the file system objects like regular files, directories, symbolic links, etc. Advanced Security Supports POSIX ACLs and SELinux in addition to the traditional file access permission model. Quotas Supports user and group quotas. Journaling Supports both ordered and writeback data journaling modes to provide file system consistency in the event of power failure or system crash. Endian and Architecture neutral Supports a cluster of nodes with mixed architectures. Allows concurrent mounts on nodes running 32-bit and 64-bit, little-endian (x86, x86_64, ia64) and big-endian (ppc64) architectures. In-built Cluster-stack with DLM Includes an easy to configure, in-kernel cluster-stack with a distributed lock manager. Buffered, Direct, Asynchronous, Splice and Memory Mapped I/Os Supports all modes of I/Os for maximum flexibility and performance. Comprehensive Tools Support Provides a familiar EXT3-style tool-set that uses similar parameters for ease-of-use. The filesystem was distributed for Linux distributions in separate RPM form and this had to be built for every single kernel errata release or every updated kernel provided by the vendor. We provided builds from Oracle for Oracle Linux and all kernels released by Oracle and for Red Hat Enterprise Linux. SuSE provided the modules directly for every kernel they shipped. With the introduction of the Unbreakable Enterprise Kernel for Oracle Linux and our interest in reducing the overhead of building filesystem modules for every minor release, we decide to make OCFS2 available as part of UEK. There was no more need for separate kernel modules, everything was built-in and a kernel upgrade automatically updated the filesystem, as it should. UEK allowed us to not having to backport new upstream filesystem code into an older kernel version, backporting features into older versions introduces risk and requires extra testing because the code is basically partially rewritten. The UEK model works really well for continuing to provide OCFS2 without that extra overhead. Because the RHEL kernel did not contain OCFS2 as a kernel module (it is in the source tree but it is not built by the vendor in kernel module form) we stopped adding the extra packages to Oracle Linux and its RHEL compatible kernel and for RHEL. Oracle Linux customers/users obviously get OCFS2 included as part of the Unbreakable Enterprise Kernel, SuSE customers get it by SuSE distributed with SLES and Red Hat can decide to distribute OCFS2 to their customers if they chose to as it's just a matter of compiling the module and making it available. OCFS2 today, in the mainline kernel is pretty much feature complete in terms of integration with every filesystem feature Linux offers and it is still actively maintained with Joel Becker being the primary maintainer. Since we use OCFS2 as part of Oracle VM, we continue to look at interesting new functionality to add, REFLINK was a good example, and as such we continue to enhance the filesystem where it makes sense. Bugfixes and any sort of code that goes into the mainline Linux kernel that affects filesystems, automatically also modifies OCFS2 so it's in kernel, actively maintained but not a lot of new development happening at this time. We continue to fully support OCFS2 as part of Oracle Linux and the Unbreakable Enterprise Kernel and other vendors make their own decisions on support as it's really a Linux cluster filesystem now more than something that we provide to customers. It really just is part of Linux like EXT3 or BTRFS etc, the OS distribution vendors decide. Do not confuse OCFS2 with ACFS (ASM cluster Filesystem) also known as Oracle Cloud Filesystem. ACFS is a filesystem that's provided by Oracle on various OS platforms and really integrates into Oracle ASM (Automatic Storage Management). It's a very powerful Cluster Filesystem but it's not distributed as part of the Operating System, it's distributed with the Oracle Database product and installs with and lives inside Oracle ASM. ACFS obviously is fully supported on Linux (Oracle Linux, Red Hat Enterprise Linux) but OCFS2 independently as a native Linux filesystem is also, and continues to also be supported. ACFS is very much tied into the Oracle RDBMS, OCFS2 is just a standard native Linux filesystem with no ties into Oracle products. Customers running the Oracle database and ASM really should consider using ACFS as it also provides storage/clustered volume management. Customers wanting to use a simple, easy to use generic Linux cluster filesystem should consider using OCFS2. To learn more about OCFS2 in detail, you can find good documentation on http://oss.oracle.com/projects/ocfs2 in the Documentation area, or get the latest mainline kernel from http://kernel.org and read the source. One final, unrelated note - since I am not always able to publicly answer or respond to comments, I do not want to selectively publish comments from readers. Sometimes I forget to publish comments, sometime I publish them and sometimes I would publish them but if for some reason I cannot publicly comment on them, it becomes a very one-sided stream. So for now I am going to not publish comments from anyone, to be fair to all sides. You are always welcome to email me and I will do my best to respond to technical questions, questions about strategy or direction are sometimes not possible to answer for obvious reasons.

  Read the article

• C#/.NET Little Wonders: The Useful But Overlooked Sets

  - by James Michael Hare

  Once again we consider some of the lesser known classes and keywords of C#.  Today we will be looking at two set implementations in the System.Collections.Generic namespace: HashSet<T> and SortedSet<T>.  Even though most people think of sets as mathematical constructs, they are actually very useful classes that can be used to help make your application more performant if used appropriately. A Background From Math In mathematical terms, a set is an unordered collection of unique items.  In other words, the set {2,3,5} is identical to the set {3,5,2}.  In addition, the set {2, 2, 4, 1} would be invalid because it would have a duplicate item (2).  In addition, you can perform set arithmetic on sets such as: Intersections: The intersection of two sets is the collection of elements common to both.  Example: The intersection of {1,2,5} and {2,4,9} is the set {2}. Unions: The union of two sets is the collection of unique items present in either or both set.  Example: The union of {1,2,5} and {2,4,9} is {1,2,4,5,9}. Differences: The difference of two sets is the removal of all items from the first set that are common between the sets.  Example: The difference of {1,2,5} and {2,4,9} is {1,5}. Supersets: One set is a superset of a second set if it contains all elements that are in the second set. Example: The set {1,2,5} is a superset of {1,5}. Subsets: One set is a subset of a second set if all the elements of that set are contained in the first set. Example: The set {1,5} is a subset of {1,2,5}. If We’re Not Doing Math, Why Do We Care? Now, you may be thinking: why bother with the set classes in C# if you have no need for mathematical set manipulation?  The answer is simple: they are extremely efficient ways to determine ownership in a collection. For example, let’s say you are designing an order system that tracks the price of a particular equity, and once it reaches a certain point will trigger an order.  Now, since there’s tens of thousands of equities on the markets, you don’t want to track market data for every ticker as that would be a waste of time and processing power for symbols you don’t have orders for.  Thus, we just want to subscribe to the stock symbol for an equity order only if it is a symbol we are not already subscribed to. Every time a new order comes in, we will check the list of subscriptions to see if the new order’s stock symbol is in that list.  If it is, great, we already have that market data feed!  If not, then and only then should we subscribe to the feed for that symbol. So far so good, we have a collection of symbols and we want to see if a symbol is present in that collection and if not, add it.  This really is the essence of set processing, but for the sake of comparison, let’s say you do a list instead: 1: // class that handles are order processing service 2: public sealed class OrderProcessor 3: { 4: // contains list of all symbols we are currently subscribed to 5: private readonly List<string> _subscriptions = new List<string>(); 6:  7: ... 8: } Now whenever you are adding a new order, it would look something like: 1: public PlaceOrderResponse PlaceOrder(Order newOrder) 2: { 3: // do some validation, of course... 4:  5: // check to see if already subscribed, if not add a subscription 6: if (!_subscriptions.Contains(newOrder.Symbol)) 7: { 8: // add the symbol to the list 9: _subscriptions.Add(newOrder.Symbol); 10: 11: // do whatever magic is needed to start a subscription for the symbol 12: } 13:  14: // place the order logic! 15: } What’s wrong with this?  In short: performance!  Finding an item inside a List<T> is a linear - O(n) – operation, which is not a very performant way to find if an item exists in a collection. (I used to teach algorithms and data structures in my spare time at a local university, and when you began talking about big-O notation you could immediately begin to see eyes glossing over as if it was pure, useless theory that would not apply in the real world, but I did and still do believe it is something worth understanding well to make the best choices in computer science). Let’s think about this: a linear operation means that as the number of items increases, the time that it takes to perform the operation tends to increase in a linear fashion.  Put crudely, this means if you double the collection size, you might expect the operation to take something like the order of twice as long.  Linear operations tend to be bad for performance because they mean that to perform some operation on a collection, you must potentially “visit” every item in the collection.  Consider finding an item in a List<T>: if you want to see if the list has an item, you must potentially check every item in the list before you find it or determine it’s not found. Now, we could of course sort our list and then perform a binary search on it, but sorting is typically a linear-logarithmic complexity – O(n * log n) - and could involve temporary storage.  So performing a sort after each add would probably add more time.  As an alternative, we could use a SortedList<TKey, TValue> which sorts the list on every Add(), but this has a similar level of complexity to move the items and also requires a key and value, and in our case the key is the value. This is why sets tend to be the best choice for this type of processing: they don’t rely on separate keys and values for ordering – so they save space – and they typically don’t care about ordering – so they tend to be extremely performant.  The .NET BCL (Base Class Library) has had the HashSet<T> since .NET 3.5, but at that time it did not implement the ISet<T> interface.  As of .NET 4.0, HashSet<T> implements ISet<T> and a new set, the SortedSet<T> was added that gives you a set with ordering. HashSet<T> – For Unordered Storage of Sets When used right, HashSet<T> is a beautiful collection, you can think of it as a simplified Dictionary<T,T>.  That is, a Dictionary where the TKey and TValue refer to the same object.  This is really an oversimplification, but logically it makes sense.  I’ve actually seen people code a Dictionary<T,T> where they store the same thing in the key and the value, and that’s just inefficient because of the extra storage to hold both the key and the value. As it’s name implies, the HashSet<T> uses a hashing algorithm to find the items in the set, which means it does take up some additional space, but it has lightning fast lookups!  Compare the times below between HashSet<T> and List<T>: Operation HashSet<T> List<T> Add() O(1) O(1) at end O(n) in middle Remove() O(1) O(n) Contains() O(1) O(n)   Now, these times are amortized and represent the typical case.  In the very worst case, the operations could be linear if they involve a resizing of the collection – but this is true for both the List and HashSet so that’s a less of an issue when comparing the two. The key thing to note is that in the general case, HashSet is constant time for adds, removes, and contains!  This means that no matter how large the collection is, it takes roughly the exact same amount of time to find an item or determine if it’s not in the collection.  Compare this to the List where almost any add or remove must rearrange potentially all the elements!  And to find an item in the list (if unsorted) you must search every item in the List. So as you can see, if you want to create an unordered collection and have very fast lookup and manipulation, the HashSet is a great collection. And since HashSet<T> implements ICollection<T> and IEnumerable<T>, it supports nearly all the same basic operations as the List<T> and can use the System.Linq extension methods as well. All we have to do to switch from a List<T> to a HashSet<T>  is change our declaration.  Since List and HashSet support many of the same members, chances are we won’t need to change much else. 1: public sealed class OrderProcessor 2: { 3: private readonly HashSet<string> _subscriptions = new HashSet<string>(); 4:  5: // ... 6:  7: public PlaceOrderResponse PlaceOrder(Order newOrder) 8: { 9: // do some validation, of course... 10: 11: // check to see if already subscribed, if not add a subscription 12: if (!_subscriptions.Contains(newOrder.Symbol)) 13: { 14: // add the symbol to the list 15: _subscriptions.Add(newOrder.Symbol); 16: 17: // do whatever magic is needed to start a subscription for the symbol 18: } 19: 20: // place the order logic! 21: } 22:  23: // ... 24: } 25: Notice, we didn’t change any code other than the declaration for _subscriptions to be a HashSet<T>.  Thus, we can pick up the performance improvements in this case with minimal code changes. SortedSet<T> – Ordered Storage of Sets Just like HashSet<T> is logically similar to Dictionary<T,T>, the SortedSet<T> is logically similar to the SortedDictionary<T,T>. The SortedSet can be used when you want to do set operations on a collection, but you want to maintain that collection in sorted order.  Now, this is not necessarily mathematically relevant, but if your collection needs do include order, this is the set to use. So the SortedSet seems to be implemented as a binary tree (possibly a red-black tree) internally.  Since binary trees are dynamic structures and non-contiguous (unlike List and SortedList) this means that inserts and deletes do not involve rearranging elements, or changing the linking of the nodes.  There is some overhead in keeping the nodes in order, but it is much smaller than a contiguous storage collection like a List<T>.  Let’s compare the three: Operation HashSet<T> SortedSet<T> List<T> Add() O(1) O(log n) O(1) at end O(n) in middle Remove() O(1) O(log n) O(n) Contains() O(1) O(log n) O(n)   The MSDN documentation seems to indicate that operations on SortedSet are O(1), but this seems to be inconsistent with its implementation and seems to be a documentation error.  There’s actually a separate MSDN document (here) on SortedSet that indicates that it is, in fact, logarithmic in complexity.  Let’s put it in layman’s terms: logarithmic means you can double the collection size and typically you only add a single extra “visit” to an item in the collection.  Take that in contrast to List<T>’s linear operation where if you double the size of the collection you double the “visits” to items in the collection.  This is very good performance!  It’s still not as performant as HashSet<T> where it always just visits one item (amortized), but for the addition of sorting this is a good thing. Consider the following table, now this is just illustrative data of the relative complexities, but it’s enough to get the point: Collection Size O(1) Visits O(log n) Visits O(n) Visits 1 1 1 1 10 1 4 10 100 1 7 100 1000 1 10 1000   Notice that the logarithmic – O(log n) – visit count goes up very slowly compare to the linear – O(n) – visit count.  This is because since the list is sorted, it can do one check in the middle of the list, determine which half of the collection the data is in, and discard the other half (binary search).  So, if you need your set to be sorted, you can use the SortedSet<T> just like the HashSet<T> and gain sorting for a small performance hit, but it’s still faster than a List<T>. Unique Set Operations Now, if you do want to perform more set-like operations, both implementations of ISet<T> support the following, which play back towards the mathematical set operations described before: IntersectWith() – Performs the set intersection of two sets.  Modifies the current set so that it only contains elements also in the second set. UnionWith() – Performs a set union of two sets.  Modifies the current set so it contains all elements present both in the current set and the second set. ExceptWith() – Performs a set difference of two sets.  Modifies the current set so that it removes all elements present in the second set. IsSupersetOf() – Checks if the current set is a superset of the second set. IsSubsetOf() – Checks if the current set is a subset of the second set. For more information on the set operations themselves, see the MSDN description of ISet<T> (here). What Sets Don’t Do Don’t get me wrong, sets are not silver bullets.  You don’t really want to use a set when you want separate key to value lookups, that’s what the IDictionary implementations are best for. Also sets don’t store temporal add-order.  That is, if you are adding items to the end of a list all the time, your list is ordered in terms of when items were added to it.  This is something the sets don’t do naturally (though you could use a SortedSet with an IComparer with a DateTime but that’s overkill) but List<T> can. Also, List<T> allows indexing which is a blazingly fast way to iterate through items in the collection.  Iterating over all the items in a List<T> is generally much, much faster than iterating over a set. Summary Sets are an excellent tool for maintaining a lookup table where the item is both the key and the value.  In addition, if you have need for the mathematical set operations, the C# sets support those as well.  The HashSet<T> is the set of choice if you want the fastest possible lookups but don’t care about order.  In contrast the SortedSet<T> will give you a sorted collection at a slight reduction in performance.   Technorati Tags: C#,.Net,Little Wonders,BlackRabbitCoder,ISet,HashSet,SortedSet

  Read the article

• Programmatically examine DLL contents

  - by Peter Hansen

  Is it possible programmatically to discover the exported names (globals, entry points, whatever) in a Windows DLL file without implementing a parser for the binary executable file format itself? I know there are tools to do this (though no open source ones I've found), but I'm curious whether there is a Windows API to accomplish the same thing or whether such tools operate merely by examining the binary file directly. I suspect there is an API for .NET libraries: if that's the case then is there a similar one for native DLLs? Edit: http://stackoverflow.com/questions/1128150 is basically an exact duplicate. The answer there is roughly "there is no API, but you can hack it using LoadLibraryEx() and navigating a few resulting data structures". Edit: I was able to use the accepted answer at http://stackoverflow.com/questions/1128150 to create a quick DLL dumper with Python and ctypes that works.

  Read the article

• CFBundleVersion error even though new version is higher

  - by Rob

  When trying to upload an upgrade binary to iTunes Connect, I am getting this error message even though I have updated CFBundleVersion in my info.plist file to a higher value (and checked the package contents after building it to be sure it copied over). I have already successfully submitted a different app upgrade by doing this same method, but for whatever reason it won't work this time. Any insights or is this an iTunes bug? The binary you uploaded was invalid. The key CFBundleVersion in the Info.plist file must contain a higher version than that of the previously uploaded version.

  Read the article

• Disk Search / Sort Algorithm

  - by AlgoMan

  Given a Range of numbers say 1 to 10,000, Input is in random order. Constraint: At any point only 1000 numbers can be loaded to memory. Assumption: Assuming unique numbers. I propose the following efficient , "When-Required-sort Algorithm". We write the numbers into files which are designated to hold particular range of numbers. For example, File1 will have 0 - 999 , File2 will have 1000 - 1999 and so on in random order. If a particular number which is say "2535" is being searched for then we know that the number is in the file3 (Binary search over range to find the file). Then file3 is loaded to memory and sorted using say Quick sort (which is optimized to add insertion sort when the array size is small ) and then we search the number in this sorted array using Binary search. And when search is done we write back the sorted file. So in long run all the numbers will be sorted. Please comment on this proposal.

  Read the article

• Git-windows case sensitive file names not handled properly

  - by dhanasekar79

  We have the git bare repository in unix that has files with same name that differs only in cases. Example: GRANT.sql grant.sql When we clone the bare repository from unix in to a windows box, git status detecs the file as modified. The working tree is loaded only with grant.sql, but git status compares grant.sql and GRANT.sql and shows the file as modified in the working tree. I tried using the core.ignorecase false but the result is the same. Is there any way to rix this issue.

  Read the article

• FILESTREAM/FILETABLE Clarifications for Implementation

  - by user1209734

  Recently our team was looking at FILESTREAM to expand the capabilities of our proprietary application. The main purpose of this app is managing the various PDFS, Images and documents to all of the parts we manufacture. Our ASP application uses a few third party tools to allow viewing of these files. We currently have 980GB of data on the Fileserver. We have around 200GB of Binary data in SQL Server that we would like to extract since it is not performing well hence FILESTREAM seems to be a good compromise to the two major data storage/access issues. A few things are not exactly clear to us: FILESTREAM Can or Cannot store its data on a drive that is not locally attached. We already have a File Server with a RAID 10 (1.5TB drives). This server stores all of the documents right now, would we have to move these drives to the SQL Server for FILESTREAM? That would be a tough bullet to bite since the server also is doubling as the Application Server (Two VMs on one physical server). FILETABLE stores the common metadata about the files but where is the Full Text part of it stored to allow searching of files like doc/docx? Is this separate? Are you able to freely add criteria to this to search by? If so any links to clarify would be appreciated. Can FILETABLE be referenced in another table with a foreign key? Thank you in advance EDIT: For those having these questions this web video covered everything and more in terms of explaining filestream from 2008 to 2012 and the cavets to consider (I would seriously rep him if I could): http://channel9.msdn.com/Events/TechDays/Techdays-2012-the-Netherlands/2270 In conclusion we will not be using FILESTREAM as it would be way to huge of an upsurge to accommodate for investment. EDIT 2: Update to #1 - After carefully assessing FileTable in addition to FILESTREAM we got a winning combination. We did have to move the files over to the new server (wasn't to painful since they were on the same VM).It honestly took more time to write an extraction tool to dump the binary data within SQL to the File System. Update to #2 - This was seperate but again Bob had an excellent webinar explaining this: http://channel9.msdn.com/Events/TechEd/Europe/2012/DBI411 Update to #3 - Using TFT inheritance we recycled the Docs table we had (minus the huge binary blobs) which required very little changes in our legacy apps. This was a huge upshot for the developer team.

  Read the article

• Mercurial extensions not working in Windows 7 x64?

  - by Samuel Meacham

  We are test driving Mercurial at work. We don't want to have to enter our user/pass each time we interact with a repository, so we set up the mercurial_keyring extension. We: Installed Python 2.6.5 (32 or 64 bit, depending on the system) Installed setuptools (for easy_install.exe) easy_install keyring easy_install mercurial_keyring And then made the appropriate changes to %userprofile%/mercurial.ini in the [auth] section. It works fine on my colleague's computer (32bit xp sp3), but it does not work on my machine (Windows 7 Ultimate x64). Also noteworthy, the setuptools had to be built from source on Win 7 x64 (python setup.py bdist_wininst, then run the resulting setuptools-0.6c11.win-amd64.exe). Using just hg.exe from the Mercurial 1.5 binary installation (the .msi), I get this error when I run hg.exe: * failed to import extension mercurial_keyring: No module named mercurial_keyring I tried to change my mercurial.ini, to specify the path to the mercurial_keyring.py file, instead of having mercurial find it (since it's in the PYTHONPATH). Old: [extensions] mercurial_keyring = New: [extensions] mercurial_keyring = c:/mercurial/extensions/mercurial_keyring.py The error changes to: abort: could not import module keyring! So while providing the path to the mercurial_keyring extension works, the dependent keyring module still cannot be found. After further investigation, it appears that NO extensions work. They all produce the error: * failed to import extension [extension name]: No module named [module name] It appears that when running hg.exe, it is not aware of PYTHONPATH. I have tried: Python 2.6.5 32 bit Python 2.6.5 64 bit Building Mercurial 1.5 from source with MinGW Building Mercurial 1.5 from source with MSVC9 Using hg.exe from the 1.5 binary dist (.msi) Using the hg.py in c:\python26\scripts when building from source Various configurations in %userprofile%/mercurial.ini Using setuptools (easy_install.exe) to install keyring and mercurial_keyring Building keyring and mercurial_keyring from source (python setup.py bdist_wininst) Nothing works. The closest I've got is using hg.py when building from source. It at least doesn't give me errors, and actually creates %userprofile%/wincrypto_pass.cfg when I enter my credentials. But on subsequent requests, it doesn't enter the credentials automatically. It prompts me for them again. Interestingly, TortoiseHG is using the keyring. I just can't get it to work on the command line. I think something is going on with Win 7 x64 that is preventing mercurial (hg.exe) from seeing the PYTHONPATH, so it can't find any of the installed modules. Does anyone have extensions working in Win 7 x64? Specifically with the binary installation of mercurial (not hg.py)?

  Read the article

• How to build 64-bit Python on OS X 10.6 -- ONLY 64 bit, no Universal nonsense

  - by ssteiner

  I just want to build this on my development machine -- the binary install from Python.org is still 32 bits and installing extensions (MySQLdb, for example) is driving me nuts with trying to figure out the proper flags for each and every extension. Clarification: I did NOT replace the system Python, I just installed the Python.org binary into its normal place at /Library/..., not /System/Library/.... Everything else seems to build 64 bit by default, and the default Python 2.6.1 was 64 bit (before I replaced it with the Python.org build figuring it was a direct replacement)` I just want a 64 bit only build that will run on my one machine without any cruft. Does anyone have a simple answer? Thanks much, [email protected]

  Read the article

• Design for a machine learning artificial intelligence framework

  - by Lirik

  This is a community wiki which aims to provide a good design for a machine learning/artificial intelligence framework (ML/AI framework). Please contribute to the design of a language-agnostic framework which would allow multiple ML/AI algorithms to be plugged into a single framework which: runs the algorithms with a user-specified data set. facilitates learning, qualification, and classification. allows users to easily plug in new algorithms. can aggregate or create an ensemble of the existing algorithms. can save/load the progress of the algorithm (i.e. save the network and weights of a neural network, save the tree of a decision tree, etc.). What is a good design for this sort of ML/AI framework?

  Read the article

• Open source embedded filesystem (or single file virtual filesystem, or structured storage) library f

  - by Ioan

  I'm not sure what the "general" name of something like this might be. I'm looking for a library that gives me a file format to store different types of binary data in an expanding single file. open source, non-GPL (LGPL ok) C interface the file format is a single file multiple files within using a POSIX-like file API (or multiple "blobs" within using some other API) file/structure editing is done in-place reliable first, performant second Examples include: the virtual drives of a virtual machine whefs HDF CDF NetCDF Problems with the above: whefs doesn't appear to be very mature, but best describes what I'm after HDF, CDF, NetCDF are usable (also very reliable and fast), but they're rather complicated and I'm not entirely convinced of their support for opaque binary "blobs" Edit: Forgot to mention, one other relevant question: http://stackoverflow.com/questions/1361560/simple-virtual-filesystem-in-c-c Another similar question: http://stackoverflow.com/questions/374417/is-there-an-open-source-alternative-to-windows-compound-files Edit: Added condition of in-place editing.

  Read the article

• ORA-12705 with OracleXE & Windows 7 & GlassFish

  - by bao

  I hate this problem... pls help! I have: GlassFish v3 (build 74.2) Windows 7 Pro english Oracle XE 10.2.0 settings: SQL select * from nls_database_parameters; PARAMETER VALUE NLS_LANGUAGE AMERICAN NLS_TERRITORY AMERICA NLS_CURRENCY $ NLS_ISO_CURRENCY AMERICA NLS_NUMERIC_CHARACTERS ., NLS_CHARACTERSET WE8MSWIN1252 NLS_CALENDAR GREGORIAN NLS_DATE_FORMAT DD-MON-RR NLS_DATE_LANGUAGE AMERICAN NLS_SORT BINARY NLS_TIME_FORMAT HH.MI.SSXFF AM PARAMETER VALUE NLS_TIMESTAMP_FORMAT DD-MON-RR HH.MI.SSXFF AM NLS_TIME_TZ_FORMAT HH.MI.SSXFF AM TZR NLS_TIMESTAMP_TZ_FORMAT DD-MON-RR HH.MI.SSXFF AM TZR NLS_DUAL_CURRENCY $ NLS_COMP BINARY NLS_LENGTH_SEMANTICS BYTE NLS_NCHAR_CONV_EXCP FALSE NLS_NCHAR_CHARACTERSET AL16UTF16 NLS_RDBMS_VERSION 10.2.0.1.0 20 rows selected. SQL HOST ECHO %NLS_LANG% AMERICAN_AMERICA.WE8MSWIN1252 NLS_LANG in registry is AMERICAN_AMERICA.WE8MSWIN1252 ORACLE_HOME in registry is C:\oraclexe\app\oracle\product\10.2.0\server I am creating Connection pool in Glassfish admin web GUI, trying to ping.. This error in log: [#|2010-05-15T23:19:26.958+0400|WARNING|glassfishv3.0|javax.enterprise.resource.resourceadapter.com.sun.enterprise.connectors.service|_ThreadID=29;_ThreadName=Thread-1;|RAR8054: Exception while creating an unpooled [test] connection for pool [ phut ], Connection could not be allocated because: ORA-00604: error occurred at recursive SQL level 1 ORA-12705: Cannot access NLS data files or invalid environment specified |#] HOW TO FIX??

  Read the article

• Is there any good reason to use <rtexprvalue>false</rtexprvalue> in JSP tags?

  - by Superfilin

  Is there any good reason to disallow scriptlet or EL expression to be inserted as attribute value? Let's say we have tag: <tag> <name>mytag</name> <tag-class>org.apache.beehive.netui.tags.tree.Tree</tag-class> <attribute> <name>attr</name> <required>false</required> <rtexprvalue>false</rtexprvalue> <type>boolean</type> </attribute> </tag> What could be a good reason for dissallowing the below? <my:mytag attr="${setting}" />

  Read the article

• WPF TreeView drag and drop using preview

  - by imekon

  I'm handling drag and drop events in a TreeView using PreviewMouseDown, PreviewMouseMove and PreviewMouseUp, however, there is an issue. In my PreviewMouseDown handler, I set everything ready in case there's a drag started (detected in the Move event), however I set e.Handled = true. This means that standard selection events don't get generated on my tree! What I want to be able to do in my Up event is to invoke the standard treeview selection changed event - except I cannot call events outside of the tree. So what's the correct way to do this? I have tried using the standard MouseDown, MouseMove and MouseUp events however there's an issue with messing up my multiple selection feature that means I need to use the Preview version of those events.

  Read the article

• MIPS (or SPIM): Loading floating point numbers...

  - by James

  Hey hey, I am working on a little mini compiler while trying to learn some MIPS here. Here's my issue: MIPS has an instruction li (load immediate) which would work like this li $5,100 which would load 100 into register 5. However, I need to load floats into registers right now and am struggling with figuring out a way to do it...since li $5,2.5 does not work. Anyone have any advice? I am working in C, I was thinking I could somehow get the integer representation of the float I am working with (i.e. so the floats binary representation == the ints binary representation) then load the "integer" into the register and treat it like a float from then on. Maybe its too late but Im stuck right now.

  Read the article

• Design for a machine learning artificial intelligence framework (community wiki)

  - by Lirik

  This is a community wiki which aims to provide a good design for a machine learning/artificial intelligence framework (ML/AI framework). Please contribute to the design of a language-agnostic framework which would allow multiple ML/AI algorithms to be plugged into a single framework which: runs the algorithms with a user-specified data set. facilitates learning, qualification, and classification. allows users to easily plug in new algorithms. can aggregate or create an ensemble of the existing algorithms. can save/load the progress of the algorithm (i.e. save the network and weights of a neural network, save the tree of a decision tree, etc.). What is a good design for this sort of ML/AI framework?

  Read the article

• iPhone Feedback Service with PHP

  - by Anish

  HI All, Has anybody been able to extract the device tokens from the binary data that iPhone APNS feedback service returns using PHP? I am looking for something similar to what is been implementented using python here http://www.google.com/codesearch/p?hl=en&sa=N&cd=2&ct=rc#m5eOMDWiKUs/APNSWrapper/%5F%5Finit%5F%5F.py&q=feedback.push.apple.com As per the Apple documentation, I know that the first 4 bytes are timestamp, next 2 bytes is the length of the token and rest of the bytes are the actual token in binary format. (http://developer.apple.com/IPhone/library/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/CommunicatingWIthAPS/CommunicatingWIthAPS.html#//apple%5Fref/doc/uid/TP40008194-CH101-SW3) I am successfully able to extract the timestamp from the data feedback service returns, but the device token that I get after i convert to hexadecimal using the PHP's built in method bin2hex() is actually different than original device token. I am doing something silly in the conversion. Can anybody help me out if they have already implemented APNS feedback service using PHP? TIA, -Anish

  Read the article

• Serialize struct with pointers to NSData

  - by leolobato

  Hey guys, I need to add some kind of archiving functionality to a Objective-C Trie implementation (NDTrie on github), but I have very little experience with C and it's data structures. struct trieNode { NSUInteger key; NSUInteger count, size; id object; __strong struct trieNode ** children; __strong struct trieNode * parent; }; @interface NDTrie (Private) - (struct trieNode*)root; @end What I need is to create an NSData with the tree structure from that root - or serialize/deserialize the whole tree some other way (conforming to NSCoding?), but I have no clue how to work with NSData and a C struct containing pointers. Performance on deserializing the resulting object would be crucial, as this is an iPhone project and I will need to load it in the background every time the app starts. What would be the best way to achieve this? Thanks!

  Read the article

• Is there a different between boost iostream mapped file and boost interprocess mapped file?

  - by Yijinsei

  hey guys, want to create a mapped binary file into memory, however I not sure how to create the file to mapped into the system. I read documentation several times and realize there is 2 mapped file, one in iostream and the other in interprocess. Do you guys have any idea on how to create a mapped file into shared memory. I trying to allow multi thread program to read an array of large double written in a binary file format. Also what is the different between the mapped file in iostream and interprocess?

  Read the article

• Extending Perforce to use a custom content diff tool for certain file extensions

  - by Fraser Graham

  I have various custom binary files stored in perforce and for many of the file types I have built a custom diff tool to show the content creators a diff of the actual changes to the file. E.g. If the file holds simple key value pairs as a compressed binary blob the diff tool would load each version into an in memory format and generate a list of additions, deletions and edits to the file presented in a nice clean report view. Much like the built in image diff tool in P4V i'd like to be able to use my own diff tool for certain file extensions within my depot and allow the users to use the existing P4V interface to pick revisions to diff between and examine history. So, I am aware you can write add-ins to P4V but I can't find any documentation on it and I'd like to know if this kind of extension functionality is available in P4V and how to use it?

  Read the article

• Storing a large list in isolatedStorage on WP7

  - by Ra

  I'm storing a List with around 3,000 objects in Isolatedstorage using Xml serialize. It takes too long to deserialize this and I was wondering if you have any recommendations to speed it up. The time is tolerable to deserialize up to 500 objects, but takes forever to deserialize 3000. Does it take longer just on the emulator and will be faster on the phone? I did a whole bunch of searching and some article said to use a binary stream reader, but I can't find it. Whether I store in binary or xml doesn't matter, I just want to persist the List. I don't want to look at asynchronous loading just yet...

  Read the article

• Setting treeview background color in VB6 has a flaw - help?

  - by RenMan

  I have successfully implemented this method of using the Win32 API to set the background color of a treeview in VB 6: http://support.microsoft.com/kb/178491 However, one thing goes wrong: when you expand the tree nodes more than two levels deep, the area to the left of (and sometimes under) the inner plus [+] and minus [-] signs is still white. Does anyone know how to get this area to the correct background color, too? Note: I'm also setting the BackColor of each node, and also the BackColor of the treeview's imagelist. Here's my version of the code: Public Sub TreeView_SetBackgroundColor(TreeView As MSComctlLib.TreeView, BackgroundColor As Long) Dim lStyle As Long, Node As MSComctlLib.Node For Each Node In TreeView.Nodes Node.BackColor = BackgroundColor Next TreeView.ImageList.BackColor = BackgroundColor Call SendMessage( _ TreeView.hwnd, _ TVM_SETBKCOLOR, _ 0, _ ByVal BackgroundColor) 'Now reset the style so that the tree lines appear properly. lStyle = GetWindowLong(TreeView.hwnd, GWL_STYLE) Call SetWindowLong(TreeView.hwnd, GWL_STYLE, lStyle - TVS_HASLINES) Call SetWindowLong(TreeView.hwnd, GWL_STYLE, lStyle) End Sub

  Read the article

• Is there something like bsdiff/Courgette for jar files?

  - by Ken Liu

  Google uses bsdiff and Courgette for patching binary files like the Chrome distribution. Do any similar tools exist for patching jar files? I am updating jar files remotely over a bandwidth-limited connection and would like to minimize the amount of data sent. I do have some control over the client machine to some extent (i.e. I can run scripts locally) and I am guaranteed that the target application will not be running at the time. I know that I can patch java applications by putting updated class files in the classpath, but I would prefer a cleaner method for doing updates. It would be good if I could start with the target jar file, apply a binary patch, and then wind up with an updated jar file that is identical (bitwise) to the new jar (from which the patch was created).

  Read the article

• LVDiff not working in Git

  - by Tanner

  I'm trying to get lvdiff from meta-diff suite to work with Git. My .gitconfig looks like this: [gui] recentrepo = C:/Users/Tanner/Desktop/FIRST 2010 Beta/Java/LoganRover [user] name = Tanner Smith email = [email protected] [merge "labview"] name = LabVIEW 3-Way Merge driver = 'C:/Program Files/National Instruments/Shared/LabVIEW Merge/LVMerge.exe' 'C:/Program Files/National Instruments/LabVIEW 8.6/LabVIEW.exe' %O %B %A %A recursive = binary [diff "lvdiff"] #command = 'C:/Program Files/meta-diff suite/lvdiff.exe' external = C:/Users/Tanner/Desktop/FIRST 2010 Beta/lvdiff.sh [core] autocrlf = true lvdiff.sh looks like this: #!/bin/sh "C:/Program Files/meta-diff suite/lvdiff.exe" "$2" "%5" | cat And my .gitattributes file looks like this: #Use a cusstom driver to merge LabVIEW files *.vi merge=labview #Use lvdiff as the externel diff program for LabVIEW files *.vi diff=lvdiff But everytime I do a diff, all Git returns is: diff --git a/Build DashBoard Data.vi b/Build DashBoard Data.vi index fd50547..662237f 100644 Binary files a/Build DashBoard Data.vi and b/Build DeashBoard Data.vi differ It is like it is not using it or even recognizing my changes. Any ideas?

  Read the article

• How do I use the iPhone Simulator in 3.2 (not iPad Simulator)

  - by JustinXXVII

  I'm fixing my app to be a universal binary. Testing on the simulator seems to default to the iPad. For small corrections like checking orientations and small UI updates, the only way I can find to get the iPhone version is to plug in my iPhone and build and run on device. Loading the debugger takes valuable time, when running on simulator is so much faster for this kind of work. Can I set the simulator to default to iPhone for this? Setting it to 3.1.3 doesn't work because of the 3.2 code I have in the binary for the iPad.

  Read the article

< Previous Page | 98 99 100 101 102 103 104 105 106 107 108 109  | Next Page >