Search Results

Search found 1246 results on 50 pages for 'compression'.

Page 15/50 | < Previous Page | 11 12 13 14 15 16 17 18 19 20 21 22  | Next Page >

• Archive format & tool for large amounts of data (50gb+)

  - by marcusstarnes

  I only realised this afternoon that the ZIP format has a limit of what appears to be around 20gb. I am trying to automate an archive process (using Automate) to zip/rar/whatever a collection of folders/files on one of my disks. It always appeared to bomb out with an incomplete archive at about 20gb. So I tried using WinRAR and doing it manually as a ZIP file, but it told me of the limit. So, I was wondering, what is a recommended zip format (and tool for accomplishing the task) for archiving up a large amount of data (around 50gb)?

  Read the article

• What’s a good way to extract .rar files on the Mac?

  - by Paul D. Waite

  Anything built-in? (E.g. command line tools.) Any well-regarded software? Update: excellent suggestions, thanks guys. I like The Unarchiver, but vote up your favourite, or add it as an answer if it isn’t there.

  Read the article

• Compression for custom mime type works on IIS 7.5, but not IIS 7.0?

  - by Doogal

  I've managed to set up compression for a custom mime type on IIS 7.5 with no problem. Add the mime type to IIS then add it to the httpCompression element in applicationHost.config. But when I do the same thing on IIS 7, the particular mime type is never compressed. This isn't a problem with compression in general since other mime types are compressed correctly. As far as I can tell, IIS 7 and IIS7.5 are configured in exactly the same way. Does IIS7 behave differently and do I need to do something else to get it working? I've setup failed request tracing and get a NO_MATCHING_CONTENT_TYPE error during compression but I can't figure out what else I need to do to tell IIS about my mime type

  Read the article

• How can I turn on DynamicCompression feature of IIS programmatically?

  - by LockeVN

  I'm making an installer program for my web application. My web application uses CSS and JS heavily, so I want to enable both Static and Dynamic HttpCompression for IIS7/7.5. It needs 2 steps: I can modified the web.config, put <httpcompression> tag, it's ok. DynamicContentCompression must be turned on in Windows Feature to make httpCompression work. Static HttpCompression is enable by default in IIS7 and IIS7.5, but Dynamic HttpCompression is not enable by default (although it's available). I can do manually by turn on: Start/ControlPanel/ProgramsAndFeatures/TurnWindowsFeatures on or Off/IIS/WWW Service/Performance features/Dynamic Content Compression, but How can I programmatically turn it on that Windows Feature? I can use PowerShell, C# in my installer. Any idea how I might be able to do this? Thanks.

  Read the article

• zlib memory usage / performance. With 500kb of data.

  - by unixman83

  Is zLib Worth it? Are there other better suited compressors? I am using an embedded system. Frequently, I have only 3MB of RAM or less available to my application. So I am considering using zlib to compress my buffers. I am concerned about overhead however. The buffer's average size will be 30kb. This probably won't get compressed by zlib. Anyone know of a good compressor for extremely limited memory environments? However, I will experience occasional maximum buffer sizes of 700kb, with 500kb much more common. Is zlib worth it in this case? Or is the overhead too much to justify? My sole considerations for compression are RAM overhead of algorithm and performance at least as good as zlib.

  Read the article

• Compressing three individual jpeg pics containing temporal redundancy?

  - by michael

  I am interfacing and embedded device with a camera module that returns a single jpeg compressed frame each time I trigger it. I would like to take three successive shots (approx 1 frame per 1/4 second) and further compress the images into a single file. The assumption here is that there is a lot of temporal redundancy, therefore lots of room for more compression across the three frames (compared to sending three separate jpeg images). I will be implementing the solution on an embedded device in C without any libraries and no OS. The camera will be taking pics in an area with very little movement (no visitors or screens in the background, maybe a tree with swaying branches), so I think my assumption about redundancy is pretty solid. When the file is finally viewed on a pc/mac, I don't mind having to write something to extract the three frames (so it can be a nonstandard cluge) So I guess the actual question is: What is the best way to compress these three images together given the fact that they are already in JPEG format (it is a possibly to convert back to a raw image, but if i dont have too...)

  Read the article

• Algorithms to find longest common prefix in a sliding window.

  - by nn

  Hi, I have written a Lempel Ziv compressor and decompressor. I am seeking to improve the time to search the dictionary for a phrase. I have considered K-M-P and Boyer-Moore, but I think an algorithm that adapts to changes in the dictionary would be faster. I've been reading that binary search trees (AVL or with splays) improve the performance of compression time considerably. What I fail to understand is how to bootstrap the binary search tree and insert/remove data. I'm not actually quite sure the significance of each node in the binary search. I am searching for phrases so will each character be considered a node? Also how and what is inserted/removed from the search tree as new data enters the dictionary and old data is removed? The binary search tree sounds like a good payoff since it can adapt to the dictionary, but I'm just not quite sure of how it's used.

  Read the article

• Best method to compress JSON string in term of performance and compress radio

  - by Eric Yin

  For a JSON string, contains all kinds of settings, numbers, string etc. Total JSON string fairly fall into 10k~50K range. I want to compress it before save to database. So I wonder which compress method should I choose, I am using c# 4, I know I can choose gzip and deflate but the compression radio is not good (although speed is good). More specific, compress can be a little slow (since only once) but should be small. Decompress should be lighting fast since decompress happens lots. Please give some advice.

  Read the article

• Compressing as GZip WCF requests (SOAP and REST)

  - by Joannes Vermorel

  I have a .NET 3.5 web app hosted on Windows Azure that exposes several WCF endpoints (both SOAP and REST). The endpoints typically receive 100x more data than they serve (lot of data is upload, much fewer is downloaded). Hence, I am willing to take advantage from HTTP GZip compression but not from the server viewpoint, but rather from the client viewpoint, sending compressed requests (returning compressed responses would be fine, but won't bring much gain anyway). Here is the little C# snippet used on the client side to activate WCF: var binding = new BasicHttpBinding(); var address = new EndpointAddress(endPoint); _factory = new ChannelFactory<IMyApi>(binding, address); _channel = _factory.CreateChannel(); Any idea how to adjust the behavior so that compressed HTTP requests can be made?

  Read the article

• How does git save space and is fast at the same time?

  - by eSKay

  I just saw the first git tutorial at http://blip.tv/play/Aeu2CAI How does git store all the versions of all the files and still be more economical in space than subversion which saves only the latest version of the code? I know this can be done using compression but that would be at the cost of speed, but this also says that git is much faster (though where is gains the max is the fact that most of its operations are offline). So, my guess is that git compresses data extensively it is still faster because uncompression + work is still faster than network_fetch + work Am I correct? even close?

  Read the article

• Storage for large gridded datasets

  - by nullglob

  I am looking for a good storage format for large, gridded datasets. The application is meteorology, and we would prefer a format that is common within this field (to help exchange data with others). I don't need to deal with special data structures, and there should be a Fortran API. I am currently considering HDF5, GRIB2 and NetCDF4. How do these formats compare in terms of data compression? What are their main limitations? How steep is the learning curve? Are there any other storage formats worth investigating? I have not found a great deal of material outlining the differences and pros/cons of these formats (there is one relevant SO thread, and a presentation comparing GRIB and NetCDF).

  Read the article

• searching within a compressed sorted fixed width file

  - by user275455

  Assume I have a regular compressed fixed width file that is sorted on one of the fields. Given that I know the length of the records, I can use lseek to implement a binary search to records with fields that match a given value without having to read the entire file. Now the difficulty is that the file is gzipped. Is it possible to do this without completely inflating the file? If not with gzip. is there any compression that supports this kind of behavior?

  Read the article

• How to efficiently deal with a large amount of HTML5 canvas pixel data over websockets

  - by user730569

  Using imageData = context.getImageData(0, 0, width, height); JSON.stringify(imageData.data); I grab the pixel data, convert it to a string, and then send it over the wire via websockets. However, this string can be pretty large, depending on the size of the canvas object. I tried using the compression technique found here: JavaScript implementation of Gzip but socket.io throws the error Websocket message contains invalid character(s). Is there an effective way to compress this data so that it can be sent over websockets?

  Read the article

• How can I compress jpeg images in Java without losing any metadata in that image?

  - by guitarpoet

  I want compress jpeg files using Java. I do it like this: Read the image as BufferedImage Write the image to another file with compression rate. OK, that seems easy, but I find the ICC color profile and the EXIF information are gone in the new file and the DPI of the image is dropped from 240 to 72. It looks different from the origin image. I use a tool like preview in OS X. It can perfectly change the quality of the image without affecting other information. Can I done this in Java? At least keep the ICC color profile and let the image color look the same as the origin photo?

  Read the article

• Compressing digitalized document images

  - by Adabada

  Hello, We are now required by law to digitalize all the financial documents in our company and submit them to evaluations every 3 months. Since this is sensitive data we decided to take matters into our own hands and build some sort of digital data archiver. The tool works perfectly, but after 7 months of usage we are begining to worry about the disk space used by these images. Here some info on the amount of documents digitalized: 15K documents scanned and archived per day, with final PNG size of +- 860KB: 15 000 * 860 kilobits = 1.53779984 gigabytes 30 days of work per month: 1.53779984 gigabytes * 30 = 46.1339952 gigabytes Expectation of disk space usage after 1 year: 46.1339952 gigabytes * 12 = 553.607942 gigabytes So far we're at 424 gigabytes of disk space used, without counting backup. We're using PNG as image format, but I would like to know if anyone have any advice on a better compression algorithm for images or alternative strategies for compressing the PNG's even more or even better ways to archive images as to save disk space. Any help would be appreciated, thanks.

  Read the article

• how to compress a PNG image using Java

  - by 116213060698242344024

  Hi I would like to know if there is any way in Java to reduce the size of an image (use any kind of compression) that was loaded as a BufferedImage and is going to be saved as an PNG. Maybe some sort of png imagewriteparam? I didnt find anything helpful so im stuck. heres a sample how the image is loaded and saved public static BufferedImage load(String imageUrl) { Image image = new ImageIcon(imageUrl).getImage(); bufferedImage = new BufferedImage(image.getWidth(null), image.getHeight(null), BufferedImage.TYPE_INT_ARGB); Graphics2D g2D = bufferedImage.createGraphics(); g2D.drawImage(image, 0, 0, null); return bufferedImage; } public static void storeImageAsPng(BufferedImage image, String imageUrl) throws IOException { ImageIO.write(image, "png", new File(imageUrl)); }

  Read the article

• Google présente « Courgette », son algorithme de compression différentielle pour réduire la taille des mises à jour de Chrome

  Google présente « Courgette », son algorithme de compression différentielle Utilisé pour réduire la taille des mises à jour du navigateur Chrome Pour une application qui évolue aussi vite que Google Chrome, le téléchargement des nombreuses mises à jour pourrait devenir un véritable casse-tête si les utilisateurs devaient rapatrier chaque fois l'installable du navigateur (environ 10 MO) Nombre d'entre eux renâcleraient certainement à l'idée de saturer leur connexion de mises à jour volumineuses...

  Read the article

• How can I replicate Google Page Speed's lossless image compression as part of my workflow?

  - by Keefer

  I love that Google's Page Speed is able to losslessly compress a lot of my images, but I'd love to make it part of my workflow, prior to uploading a site and making it live. Is there anything I can run locally to give me the same lossless compression? I currently export images from Export For Web from Photoshop, and use a little application called PNGCrusher to reduce file size of PNGs. I'd love to find a faster way though than saving out and replacing the individual images from Page Speed's results.

  Read the article

• asp.net mvc compress stream and remove whitespace

  - by Bigfellahull

  Hi, So I am compressing my output stream via an action filter: var response = filterContext.HttpContext.Response; response.Filter = new DeflateStream(response.Filter), CompressionMode.Compress); Which works great. Now, I would also like to remove the excess whitespace present. I found Mads Kristensen's http module http://madskristensen.net/post/A-whitespace-removal-HTTP-module-for-ASPNET-20.aspx. I added the WhitespaceFilter class and added a new filter like the compression: var response = filterContext.HttpContext.Response; response.Filter = new WhitepaperFilter(response.Filter); This also works great. However, I seem to be having problems combining the two! I tried: var response = filterContext.HttpContext.Response; response.Filter = new DeflateStream(new WhitespaceFilter(response.Filter), CompressionMode.Compress); However this results in some major issues. The html gets completely messed up and sometimes I get an 330 error. It seems that the Whitespace filter write method gets called multiple times. The first time the html string is fine, but on subsequent calls its just random characters. I thought it might be because the stream had been deflated, but isnt the whitespace filter using the untouched stream and then passing the resulting stream to the DeflateStream call? Any ideas?

  Read the article

• iis7 compress dynamic content from custom handler

  - by Malloc

  I am having trouble getting dynamic content coming from a custom handler to be compressed by IIS 7. Our handler spits out json data (Content-Type: application/json; charset=utf-8) and responds to url that looks like: domain.com/example.mal/OperationName?Param1=Val1&Param2=Val2 In IIS 6, all we had to do was put the edit the MetaBase.xml and in the IIsCompressionScheme element make sure that the HcScriptFileExtensions attribute had the custom extension 'mal' included in it. Static and Dynamic compression is turned out at the server and website level. I can confirm that normal .aspx pages are compressed correctly. The only content I cannot have compressed is the content coming from the custom handler. I have tried the following configs with no success: <handlers> <add name="MyJsonService" verb="GET,POST" path="*.mal" type="Library.Web.HttpHandlers.MyJsonServiceHandlerFactory, Library.Web" /> </handlers> <httpCompression> <dynamicTypes> <add mimeType="application/json" enabled="true" /> </dynamicTypes> </httpCompression> _ <httpCompression> <dynamicTypes> <add mimeType="application/*" enabled="true" /> </dynamicTypes> </httpCompression> _ <staticContent> <mimeMap fileExtension=".mal" mimeType="application/json" /> </staticContent> <httpCompression> <dynamicTypes> <add mimeType="application/*" enabled="true" /> </dynamicTypes> </httpCompression> Thanks in advance for the help.

  Read the article

• How do I compress a Json result from ASP.NET MVC with IIS 7.5

  - by Gareth Saul

  I'm having difficulty making IIS 7 correctly compress a Json result from ASP.NET MVC. I've enabled static and dynamic compression in IIS. I can verify with Fiddler that normal text/html and similar records are compressed. Viewing the request, the accept-encoding gzip header is present. The response has the mimetype "application/json", but is not compressed. I've identified that the issue appears to relate to the MimeType. When I include mimeType="*/*", I can see that the response is correctly gzipped. How can I get IIS to compress WITHOUT using a wildcard mimeType? I assume that this issue has something to do with the way that ASP.NET MVC generates content type headers. The CPU usage is well below the dynamic throttling threshold. When I examine the trace logs from IIS, I can see that it fails to compress due to not finding a matching mime type. <httpCompression directory="%SystemDrive%\inetpub\temp\IIS Temporary Compressed Files" noCompressionForProxies="false"> <scheme name="gzip" dll="%Windir%\system32\inetsrv\gzip.dll" /> <dynamicTypes> <add mimeType="text/*" enabled="true" /> <add mimeType="message/*" enabled="true" /> <add mimeType="application/x-javascript" enabled="true" /> <add mimeType="application/json" enabled="true" /> </dynamicTypes> <staticTypes> <add mimeType="text/*" enabled="true" /> <add mimeType="message/*" enabled="true" /> <add mimeType="application/x-javascript" enabled="true" /> <add mimeType="application/atom+xml" enabled="true" /> <add mimeType="application/xaml+xml" enabled="true" /> <add mimeType="application/json" enabled="true" /> </staticTypes> </httpCompression>

  Read the article

• Reconstructing trees from a "fingerprint"

  - by awshepard

  I've done my SO and Google research, and haven't found anyone who has tackled this before, or at least, anyone who has written about it. My question is, given a "universal" tree of arbitrary height, with each node able to have an arbitrary number of branches, is there a way to uniquely (and efficiently) "fingerprint" arbitrary sub-trees starting from the "universal" tree's root, such that given the universal tree and a tree's fingerprint, I can reconstruct the original tree? For instance, I have a "universal" tree (forgive my poor illustrations), representing my universe of possibilities: Root / / / | \ \ ... \ O O O O O O O (Level 1) /|\/|\...................\ (Level 2) etc. I also have tree A, a rooted subtree of my universe Root / /|\ \ O O O O O / Etc. Is there a way to "fingerprint" the tree, so that given that fingerprint, and the universal tree, I could reconstruct A? I'm thinking something along the lines of a hash, a compression, or perhaps a functional/declarative construction? Big-O analysis (in time or space) is a plus. As a for-instance, a nested expression like: {{(Root)},{(1),(2),(3)},{(2,3),(1),(4,5)}...} representing the actual nodes present at each level in the tree is probably valid, but can it be done more efficiently?

  Read the article

• Writing a JavaScript zip code validation function

  - by mkoryak

  I would like to write a JavaScript function that validates a zip code, by checking if the zip code actually exists. Here is a list of all zip codes: http://www.census.gov/tiger/tms/gazetteer/zips.txt (I only care about the 2nd column) This is really a compression problem. I would like to do this for fun. OK, now that's out of the way, here is a list of optimizations over a straight hashtable that I can think of, feel free to add anything I have not thought of: Break zipcode into 2 parts, first 2 digits and last 3 digits. Make a giant if-else statement first checking the first 2 digits, then checking ranges within the last 3 digits. Or, covert the zips into hex, and see if I can do the same thing using smaller groups. Find out if within the range of all valid zip codes there are more valid zip codes vs invalid zip codes. Write the above code targeting the smaller group. Break up the hash into separate files, and load them via Ajax as user types in the zipcode. So perhaps break into 2 parts, first for first 2 digits, second for last 3. Lastly, I plan to generate the JavaScript files using another program, not by hand. Edit: performance matters here. I do want to use this, if it doesn't suck. Performance of the JavaScript code execution + download time. Edit 2: JavaScript only solutions please. I don't have access to the application server, plus, that would make this into a whole other problem =)

  Read the article

• How can I tell if a byte array has already been compressed?

  - by MikeG

  Hi, Can I rely on the first few bytes of data compressed using the System.IO.Compression.DeflateStream in .NET always being the same? These bytes seem to always be the 1st bytes: 237, 189, 7, 96, 28, 73, 150, 37, 38, 47 , ... I'm assuming this is some kind of header, I'd like to assume that this header is fixed and isn't going to change. Has anyone got any extra info about this? Background info (The reason I want to know this info is...) I have a load of data in a database table that could do with being made smaller. I've decided I'm going to start compressing the data and not going to bother compressing the existing data. When the data gets into my .NET code the data is a String. I'd like to be able to look at the 1st few bytes of the string and see if it has been compressed, if it has then I need to de-compress it. I was originally thinking I could convert the string to bytes and just try de-compressing the data. Then if an exception happens, I could just assume it wasn't compressed. But I think checking the header bytes would give me much better performance. Many thanks, Mike G

  Read the article

• Dealing with large number of text strings

  - by Fadrian

  My project when it is running, will collect a large number of string text block (about 20K and largest I have seen is about 200K of them) in short span of time and store them in a relational database. Each of the string text is relatively small and the average would be about 15 short lines (about 300 characters). The current implementation is in C# (VS2008), .NET 3.5 and backend DBMS is Ms. SQL Server 2005 Performance and storage are both important concern of the project, but the priority will be performance first, then storage. I am looking for answers to these: Should I compress the text before storing them in DB? or let SQL Server worry about compacting the storage? Do you know what will be the best compression algorithm/library to use for this context that gives me the best performance? Currently I just use the standard GZip in .NET framework Do you know any best practices to deal with this? I welcome outside the box suggestions as long as it is implementable in .NET framework? (it is a big project and this requirements is only a small part of it) EDITED: I will keep adding to this to clarify points raised I don't need text indexing or searching on these text. I just need to be able to retrieve them in later stage for display as a text block using its primary key. I have a working solution implemented as above and SQL Server has no issue at all handling it. This program will run quite often and need to work with large data context so you can imagine the size will grow very rapidly hence every optimization I can do will help.

  Read the article

< Previous Page | 11 12 13 14 15 16 17 18 19 20 21 22  | Next Page >