Search Results

Search found 18899 results on 756 pages for 'python c extension'.

Page 752/756 | < Previous Page | 748 749 750 751 752 753 754 755 756  | Next Page >

• Should we hire someone who writes C in Perl?

  - by paxdiablo

  One of my colleagues recently interviewed some candidates for a job and one said they had very good Perl experience. Since my colleague didn't know Perl, he asked me for a critique of some code written (off-site) by that potential hire, so I had a look and told him my concerns (the main one was that it originally had no comments and it's not like we gave them enough time). However, the code works so I'm loathe to say no-go without some more input. Another concern is that this code basically looks exactly how I'd code it in C. It's been a while since I did Perl (and I didn't do a lot, I'm more a Python bod for quick scripts) but I seem to recall that it was a much more expressive language than what this guy used. I'm looking for input from real Perl coders, and suggestions for how it could be improved (and why a Perl coder should know that method of improvement). You can also wax lyrical about whether people who write one language in a totally different language should (or shouldn't be hired). I'm interested in your arguments but this question is primarily for a critique of the code. The spec was to successfully process a CSV file as follows and output the individual fields: User ID,Name , Level,Numeric ID pax, Pax Morgan ,admin,0 gt," Turner, George" rubbish,user,1 ms,"Mark \"X-Men\" Spencer","guest user",2 ab,, "user","3" The output was to be something like this (the potential hire's code actually output this): User ID,Name , Level,Numeric ID: [User ID] [Name] [Level] [Numeric ID] pax, Pax Morgan ,admin,0: [pax] [Pax Morgan] [admin] [0] gt," Turner, George " rubbish,user,1: [gt] [ Turner, George ] [user] [1] ms,"Mark \"X-Men\" Spencer","guest user",2: [ms] [Mark "X-Men" Spencer] [guest user] [2] ab,, "user","3": [ab] [] [user] [3] Here is the code they submitted: #!/usr/bin/perl # Open file. open (IN, "qq.in") || die "Cannot open qq.in"; # Process every line. while (<IN>) { chomp; $line = $_; print "$line:\n"; # Process every field in line. while ($line ne "") { # Skip spaces and start with empty field. if (substr ($line,0,1) eq " ") { $line = substr ($line,1); next; } $field = ""; $minlen = 0; # Detect quoted field or otherwise. if (substr ($line,0,1) eq "\"") { $line = substr ($line,1); $pastquote = 0; while ($line ne "") { # Special handling for quotes (\\ and \"). if (length ($line) >= 2) { if (substr ($line,0,2) eq "\\\"") { $field = $field . "\""; $line = substr ($line,2); next; } if (substr ($line,0,2) eq "\\\\") { $field = $field . "\\"; $line = substr ($line,2); next; } } # Detect closing quote. if (($pastquote == 0) && (substr ($line,0,1) eq "\"")) { $pastquote = 1; $line = substr ($line,1); $minlen = length ($field); next; } # Only worry about comma if past closing quote. if (($pastquote == 1) && (substr ($line,0,1) eq ",")) { $line = substr ($line,1); last; } $field = $field . substr ($line,0,1); $line = substr ($line,1); } } else { while ($line ne "") { if (substr ($line,0,1) eq ",") { $line = substr ($line,1); last; } if ($pastquote == 0) { $field = $field . substr ($line,0,1); } $line = substr ($line,1); } } # Strip trailing space. while ($field ne "") { if (length ($field) == $minlen) { last; } if (substr ($field,length ($field)-1,1) eq " ") { $field = substr ($field,0, length ($field)-1); next; } last; } print " [$field]\n"; } } close (IN);

  Read the article

• Google App Engine - Secure Cookies

  - by tponthieux

  I'd been searching for a way to do cookie based authentication/sessions in Google App Engine because I don't like the idea of memcache based sessions, and I also don't like the idea of forcing users to create google accounts just to use a website. I stumbled across someone's posting that mentioned some signed cookie functions from the Tornado framework and it looks like what I need. What I have in mind is storing a user's id in a tamper proof cookie, and maybe using a decorator for the request handlers to test the authentication status of the user, and as a side benefit the user id will be available to the request handler for datastore work and such. The concept would be similar to forms authentication in ASP.NET. This code comes from the web.py module of the Tornado framework. According to the docstrings, it "Signs and timestamps a cookie so it cannot be forged" and "Returns the given signed cookie if it validates, or None." I've tried to use it in an App Engine Project, but I don't understand the nuances of trying to get these methods to work in the context of the request handler. Can someone show me the right way to do this without losing the functionality that the FriendFeed developers put into it? The set_secure_cookie, and get_secure_cookie portions are the most important, but it would be nice to be able to use the other methods as well. #!/usr/bin/env python import Cookie import base64 import time import hashlib import hmac import datetime import re import calendar import email.utils import logging def _utf8(s): if isinstance(s, unicode): return s.encode("utf-8") assert isinstance(s, str) return s def _unicode(s): if isinstance(s, str): try: return s.decode("utf-8") except UnicodeDecodeError: raise HTTPError(400, "Non-utf8 argument") assert isinstance(s, unicode) return s def _time_independent_equals(a, b): if len(a) != len(b): return False result = 0 for x, y in zip(a, b): result |= ord(x) ^ ord(y) return result == 0 def cookies(self): """A dictionary of Cookie.Morsel objects.""" if not hasattr(self,"_cookies"): self._cookies = Cookie.BaseCookie() if "Cookie" in self.request.headers: try: self._cookies.load(self.request.headers["Cookie"]) except: self.clear_all_cookies() return self._cookies def _cookie_signature(self,*parts): self.require_setting("cookie_secret","secure cookies") hash = hmac.new(self.application.settings["cookie_secret"], digestmod=hashlib.sha1) for part in parts:hash.update(part) return hash.hexdigest() def get_cookie(self,name,default=None): """Gets the value of the cookie with the given name,else default.""" if name in self.cookies: return self.cookies[name].value return default def set_cookie(self,name,value,domain=None,expires=None,path="/", expires_days=None): """Sets the given cookie name/value with the given options.""" name = _utf8(name) value = _utf8(value) if re.search(r"[\x00-\x20]",name + value): # Don't let us accidentally inject bad stuff raise ValueError("Invalid cookie %r:%r" % (name,value)) if not hasattr(self,"_new_cookies"): self._new_cookies = [] new_cookie = Cookie.BaseCookie() self._new_cookies.append(new_cookie) new_cookie[name] = value if domain: new_cookie[name]["domain"] = domain if expires_days is not None and not expires: expires = datetime.datetime.utcnow() + datetime.timedelta( days=expires_days) if expires: timestamp = calendar.timegm(expires.utctimetuple()) new_cookie[name]["expires"] = email.utils.formatdate( timestamp,localtime=False,usegmt=True) if path: new_cookie[name]["path"] = path def clear_cookie(self,name,path="/",domain=None): """Deletes the cookie with the given name.""" expires = datetime.datetime.utcnow() - datetime.timedelta(days=365) self.set_cookie(name,value="",path=path,expires=expires, domain=domain) def clear_all_cookies(self): """Deletes all the cookies the user sent with this request.""" for name in self.cookies.iterkeys(): self.clear_cookie(name) def set_secure_cookie(self,name,value,expires_days=30,**kwargs): """Signs and timestamps a cookie so it cannot be forged""" timestamp = str(int(time.time())) value = base64.b64encode(value) signature = self._cookie_signature(name,value,timestamp) value = "|".join([value,timestamp,signature]) self.set_cookie(name,value,expires_days=expires_days,**kwargs) def get_secure_cookie(self,name,include_name=True,value=None): """Returns the given signed cookie if it validates,or None""" if value is None:value = self.get_cookie(name) if not value:return None parts = value.split("|") if len(parts) != 3:return None if include_name: signature = self._cookie_signature(name,parts[0],parts[1]) else: signature = self._cookie_signature(parts[0],parts[1]) if not _time_independent_equals(parts[2],signature): logging.warning("Invalid cookie signature %r",value) return None timestamp = int(parts[1]) if timestamp < time.time() - 31 * 86400: logging.warning("Expired cookie %r",value) return None try: return base64.b64decode(parts[0]) except: return None uid=1234|1234567890|d32b9e9c67274fa062e2599fd659cc14 Parts: 1. uid is the name of the key 2. 1234 is your value in clear 3. 1234567890 is the timestamp 4. d32b9e9c67274fa062e2599fd659cc14 is the signature made from the value and the timestamp

  Read the article

• Code Golf: Finite-state machine!

  - by Adam Matan

  Finite state machine A deterministic finite state machine is a simple computation model, widely used as an introduction to automata theory in basic CS courses. It is a simple model, equivalent to regular expression, which determines of a certain input string is Accepted or Rejected. Leaving some formalities aside, A run of a finite state machine is composed of: alphabet, a set of characters. states, usually visualized as circles. One of the states must be the start state. Some of the states might be accepting, usually visualized as double circles. transitions, usually visualized as directed arches between states, are directed links between states associated with an alphabet letter. input string, a list of alphabet characters. A run on the machine begins at the starting state. Each letter of the input string is read; If there is a transition between the current state and another state which corresponds to the letter, the current state is changed to the new state. After the last letter was read, if the current state is an accepting state, the input string is accepted. If the last state was not an accepting state, or a letter had no corresponding arch from a state during the run, the input string is rejected. Note: This short descruption is far from being a full, formal definition of a FSM; Wikipedia's fine article is a great introduction to the subject. Example For example, the following machine tells if a binary number, read from left to right, has an even number of 0s: The alphabet is the set {0,1}. The states are S1 and S2. The transitions are (S1, 0) -> S2, (S1, 1) -> S1, (S2, 0) -> S1 and (S2, 1) -> S2. The input string is any binary number, including an empty string. The rules: Implement a FSM in a language of your choice. Input The FSM should accept the following input: <States> List of state, separated by space mark. The first state in the list is the start state. Accepting states begin with a capital letter. <transitions> One or more lines. Each line is a three-tuple: origin state, letter, destination state) <input word> Zero or more characters, followed by a newline. For example, the aforementioned machine with 1001010 as an input string, would be written as: S1 s2 S1 0 s2 S1 1 S1 s2 0 S1 s2 1 s2 1001010 Output The FSM's run, written as <State> <letter> -> <state>, followed by the final state. The output for the example input would be: S1 1 -> S1 S1 0 -> s2 s2 0 -> S1 S1 1 -> S1 S1 0 -> s2 s2 1 -> s2 s2 0 -> S1 ACCEPT For the empty input '': S1 ACCEPT For 101: S1 1 -> S1 S1 0 -> s2 s2 1 -> s2 REJECT For '10X': S1 1 -> S1 S1 0 -> s2 s2 X REJECT Prize A nice bounty will be given to the most elegant and short solution. Reference implementation A reference Python implementation will be published soon.

  Read the article

• Django ImageField issue with JPEG's

  - by Kieran Lynn

  I am having a major issue with PIL (Python Image Library) in Django and have jumpped through a lot of hoops and have thus far not been able to figure out what the root of the issue is. The problem essentially breaks down to not being able to upload JPEG images through the ImageField in the Django admin. But the issue is not as simple as installing libjpeg. First, I installed PIL (through Buildout) and realized once it was installed that I had not installed libjpeg because JPEG support was not available. Having not setup the server myself, I just assumed that it was not installed and I compiled libjpeg 8 from the source. This ended up in my /usr/local/lib/ directory. I cleared out my Buildout files and rebuilt everything. This time when PIL compiled I had JPEG support. But I went to the Django Admin and tried to upload a JPEG though an ImageField with no luck. I got the "Upload a valid image. The file you uploaded was either not an image or a corrupted image" error. Just as a test I opened up a the Djano shell and ran the following: > import Image > i = Image.open( "/absolute_path/file.jpg" ) > print i <JpegImagePlugin.JpegImageFile image mode=RGB size=940x375 at 0x7F908C529BD8> This runs with no errors and shows that PIL is able to open JPEG's. After doing some reading, I come across this thread: Is it possible to control which libraries apache uses? Looks like PHP also uses libjpeg and is loading before Django, and therefor loading libjpeg 6.2 before. This is show when using lsof: COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME apache2 2561 www-data mem REG 202,1 146032 639276 /usr/lib/libjpeg.so.62.0.0 So my thought is that I should be using libjpeg 6.2. So I removed libjpeg located in my /usr/local/lib directory. After rereading the PIL installation instructions, I realized that I might not have the dev/header files for libjpeg that PIL needs. So I also uninstalled libjpeg using the aptitude uninstaller (sudo aptitude remove libjpeg62). Then to ensure that I got the header files that PIL needed I installed libjpeg using aptitude: (sudo aptget install libjpeg62-dev). From here I cleaned out my Buildout directory, and reran Buildout, which in turn reinstalled PIL. Once again, I have JPEG support, now using the libjpeg62. So I go to test in the Django Admin. Still no JPEG support. So I wanted to test JPEG support in general and see if the exception was not handled, what kind of error it would throw. So in my homepage view I added the following code to open a JPEG image: import Image i = Image.open( "/absolute_path/file.jpg" ) v = i.verify() Then I pass i to the HTML view just to easily see the output. I deploy these changes to the server and restart. I am surprised not to see an error and get the following output: {{ i }} - <JpegImagePlugin.JpegImageFile image mode=RGB size=940x375 at 0x7F908C529BD8> {{ v }} - None So at this point I am really confused: Why can I successfully open a JPEG while the admin cannot? Am I missing something, is this not an issue with libjpeg? If not an issue with libjpeg, why can I upload a PNG with no issues? Any help would be much appreciated, I have been on this for 2 days debugging with no luck. Setup: 1. Rackspace Cloud Server 2. Ubuntu 10.04 3. Django 1.2.3 (Installed though Buildout) 4. PIL 1.1.7 (Installed though Buildout) 5. libjpeg 6.2 (installed through aptitude (sudo aptget install libjpeg62-dev)

  Read the article

• Qt 4.6 Adding objects and sub-objects to QWebView window object (C++ & Javascript)

  - by Cor

  I am working with Qt's QWebView, and have been finding lots of great uses for adding to the webkit window object. One thing I would like to do is nested objects... for instance: in Javascript I can... var api = new Object; api.os = new Object; api.os.foo = function(){} api.window = new Object(); api.window.bar = function(){} obviously in most cases this would be done through a more OO js-framework. This results in a tidy structure of: >>>api ------------------------------------------------------- - api Object {os=Object, more... } - os Object {} foo function() - win Object {} bar function() ------------------------------------------------------- Right now I'm able to extend the window object with all of the qtC++ methods and signals I need, but they all have 'seem' to have to be in a root child of "window". This is forcing me to write a js wrapper object to get the hierarchy that I want in the DOM. >>>api ------------------------------------------------------- - api Object {os=function, more... } - os_foo function() - win_bar function() ------------------------------------------------------- This is a pretty simplified example... I want objects for parameters, etc... Does anyone know of a way to pass an child object with the object that extends the WebFrame's window object? Here's some example code of how I'm adding the object: mainwindow.h #ifndef MAINWINDOW_H #define MAINWINDOW_H #include <QtGui/QMainWindow> #include <QWebFrame> #include "mainwindow.h" #include "happyapi.h" class QWebView; class QWebFrame; QT_BEGIN_NAMESPACE class MainWindow : public QMainWindow { Q_OBJECT public: MainWindow(QWidget *parent = 0); private slots: void attachWindowObject(); void bluesBros(); private: QWebView *view; HappyApi *api; QWebFrame *frame; }; #endif // MAINWINDOW_H mainwindow.cpp #include <QDebug> #include <QtGui> #include <QWebView> #include <QWebPage> #include "mainwindow.h" #include "happyapi.h" MainWindow::MainWindow(QWidget *parent) : QMainWindow(parent) { view = new QWebView(this); view->load(QUrl("file:///Q:/example.htm")); api = new HappyApi(this); QWebPage *page = view->page(); frame = page->mainFrame(); attachWindowObject(); connect(frame, SIGNAL(javaScriptWindowObjectCleared()), this, SLOT(attachWindowObject())); connect(api, SIGNAL(win_bar()), this, SLOT(bluesBros())); setCentralWidget(view); }; void MainWindow::attachWindowObject() { frame->addToJavaScriptWindowObject(QString("api"), api); }; void MainWindow::bluesBros() { qDebug() << "foo and bar are getting the band back together!"; }; happyapi.h #ifndef HAPPYAPI_H #define HAPPYAPI_H #include <QObject> class HappyApi : public QObject { Q_OBJECT public: HappyApi(QObject *parent); public slots: void os_foo(); signals: void win_bar(); }; #endif // HAPPYAPI_H happyapi.cpp #include <QDebug> #include "happyapi.h" HappyApi::HappyApi(QObject *parent) : QObject(parent) { }; void HappyApi::os_foo() { qDebug() << "foo called, it want's it's bar back"; }; I'm reasonably new to C++ programming (coming from a web and python background). Hopefully this example will serve to not only help other new users, but be something interesting for a more experienced c++ programmer to elaborate on. Thanks for any assistance that can be provided. :)

  Read the article

• The best cross platform (portable) arbitrary precision math library

  - by Siu Ching Pong - Asuka Kenji

  Dear ninjas / hackers / wizards, I'm looking for a good arbitrary precision math library in C or C++. Could you please give me some advices / suggestions? The primary requirements: It MUST handle arbitrarily big integers (my primary interest is on integers). In case that you don't know what the word arbitrarily big means, imagine something like 100000! (the factorial of 100000). The precision MUST NOT NEED to be specified during library initialization / object creation. The precision should ONLY be constrained by the available resources of the system. It SHOULD utilize the full power of the platform, and should handle "small" numbers natively. That means on a 64-bit platform, calculating 2^33 + 2^32 should use the available 64-bit CPU instructions. The library SHOULD NOT calculate this in the same way as it does with 2^66 + 2^65 on the same platform. It MUST handle addition (+), subtraction (-), multiplication (*), integer division (/), remainder (%), power (**), increment (++), decrement (--), gcd(), factorial(), and other common integer arithmetic calculations efficiently. Ability to handle functions like sqrt() (square root), log() (logarithm) that do not produce integer results is a plus. Ability to handle symbolic computations is even better. Here are what I found so far: Java's BigInteger and BigDecimal class: I have been using these so far. I have read the source code, but I don't understand the math underneath. It may be based on theories / algorithms that I have never learnt. The built-in integer type or in core libraries of bc / Python / Ruby / Haskell / Lisp / Erlang / OCaml / PHP / some other languages: I have ever used some of these, but I have no idea on which library they are using, or which kind of implementation they are using. What I have already known: Using a char as a decimal digit, and a char* as a decimal string and do calculations on the digits using a for-loop. Using an int (or a long int, or a long long) as a basic "unit" and an array of it as an arbitrary long integer, and do calculations on the elements using a for-loop. Booth's multiplication algorithm What I don't know: Printing the binary array mentioned above in decimal without using naive methods. Example of a naive method: (1) add the bits from the lowest to the highest: 1, 2, 4, 8, 16, 32, ... (2) use a char* string mentioned above to store the intermediate decimal results). What I appreciate: Good comparisons on GMP, MPFR, decNumber (or other libraries that are good in your opinion). Good suggestions on books / articles that I should read. For example, an illustration with figures on how a un-naive arbitrarily long binary to decimal conversion algorithm works is good. Any help. Please DO NOT answer this question if: you think using a double (or a long double, or a long long double) can solve this problem easily. If you do think so, it means that you don't understand the issue under discussion. you have no experience on arbitrary precision mathematics. Thank you in advance! Asuka Kenji

  Read the article

• Writing more efficient xquery code (avoiding redundant iteration)

  - by Coquelicot

  Here's a simplified version of a problem I'm working on: I have a bunch of xml data that encodes information about people. Each person is uniquely identified by an 'id' attribute, but they may go by many names. For example, in one document, I might find <person id=1>Paul Mcartney</person> <person id=2>Ringo Starr</person> And in another I might find: <person id=1>Sir Paul McCartney</person> <person id=2>Richard Starkey</person> I want to use xquery to produce a new document that lists every name associated with a given id. i.e.: <person id=1> <name>Paul McCartney</name> <name>Sir Paul McCartney</name> <name>James Paul McCartney</name> </person> <person id=2> ... </person> The way I'm doing this now in xquery is something like this (pseudocode-esque): let $ids := distinct-terms( [all the id attributes on people] ) for $id in $ids return <person id={$id}> { for $unique-name in distinct-values ( for $name in ( [all names] ) where $name/@id=$id return $name ) return <name>{$unique-name}</name> } </person> The problem is that this is really slow. I imagine the bottleneck is the innermost loop, which executes once for every id (of which there are about 1200). I'm dealing with a fair bit of data (300 MB, spread over about 800 xml files), so even a single execution of the query in the inner loop takes about 12 seconds, which means that repeating it 1200 times will take about 4 hours (which might be optimistic - the process has been running for 3 hours so far). Not only is it slow, it's using a whole lot of virtual memory. I'm using Saxon, and I had to set java's maximum heap size to 10 GB (!) to avoid getting out of memory errors, and it's currently using 6 GB of physical memory. So here's how I'd really like to do this (in Pythonic pseudocode): persons = {} for id in ids: person[id] = set() for person in all_the_people_in_my_xml_document: persons[person.id].add(person.name) There, I just did it in linear time, with only one sweep of the xml document. Now, is there some way to do something similar in xquery? Surely if I can imagine it, a reasonable programming language should be able to do it (he said quixotically). The problem, I suppose, is that unlike Python, xquery doesn't (as far as I know) have anything like an associative array. Is there some clever way around this? Failing that, is there something better than xquery that I might use to accomplish my goal? Because really, the computational resources I'm throwing at this relatively simple problem are kind of ridiculous.

  Read the article

• Writing a code example

  - by Stefano Borini

  I would like to have your feedback regarding code examples. One of the most frustrating experiences I sometimes have when learning a new technology is finding useless examples. I think an example as the most precious thing that comes with a new library, language, or technology. It must be a starting point, a wise and unadulterated explanation on how to achieve a given result. A perfect example must have the following characteristics: Self contained: it should be small enough to be compiled or executed as a single program, without dependencies or complex makefiles. An example is also a strong functional test if you correctly installed the new technology. The more issues could arise, the more likely is that something goes wrong, and the more difficult is to debug and solve the situation. Pertinent: it should demonstrate one, and only one, specific feature of your software/library, involving the minimal additional behavior from external libraries. Helpful: the code should bring you forward, step by step, using comments or self-documenting code. Extensible: the example code should be a small “framework” or blueprint for additional tinkering. A learner can start by adding features to this blueprint. Recyclable: it should be possible to extract parts of the example to use in your own code Easy: An example code is not the place to show your code-fu skillz. Keep it easy. helpful acronym: SPHERE. Prototypical examples of violations of those rules are the following: Violation of self-containedness: an example spanning multiple files without any real need for it. If your example is a python program, keep everything into a single module file. Don’t sub-modularize it. In Java, try to keep everything into a single class, unless you really must partition some entity into a meaningful object you need to pass around (and java mandates one class per file, if I remember correctly). Violation of Pertinency: When showing how many different shapes you can draw, adding radio buttons and complex controls with all the possible choices for point shapes is a bad idea. You de-focalize your example code, introducing code for event handling, controls initialization etc., and this is not part the feature you want to demonstrate, they are unnecessary noise in the understanding of the crucial mechanisms providing the feature. Violation of Helpfulness: code containing dubious naming, wrong comments, hacks, and functions longer than one page of code. Violation of Extensibility: badly factored code that have everything into a single function, with potentially swappable entities embedded within the code. Example: if an example reads data from a file and displays it, create a method getData() returning a useful entity, instead of opening the file raw and plotting the stuff. This way, if the user of the library needs to read data from a HTTP server instead, he just has to modify the getData() module and use the example almost as-is. Another violation of Extensibility comes if the example code is not under a fully liberal (e.g. MIT or BSD) license. Violation of Recyclability: when the code layout is so intermingled that is difficult to easily copy and paste parts of it and recycle them into another program. Again, licensing is also a factor. Violation of Easiness: Yes, you are a functional-programming nerd and want to show how cool you are by doing everything on a single line of map, filter and so on, but that could not be helpful to someone else, who is already under pressure to understand your library, and now has to understand your code as well. And in general, the final rule: if it takes more than 10 minutes to do the following: compile the code, run it, read the source, and understand it fully, it means that the example is not a good one. Please let me know your opinion, either positive or negative, or experience on this regard.

  Read the article

• Google Drive Update keeps throwing an error when I try to save a thumbnail

  - by Dano64

  The Base64 string checks out fine. I was able to export it to another website and download it again as my image. When I try to do an update using the Drive Javascript api, it just keeps returning this error: Invalidvaluefor:Notavalidbase64bytestring I also true making the string URL safe. Per this page https://google-api-client-libraries.appspot.com/documentation/drive/v2/python/latest/drive_v2.files.html Am I doing something wrong here, the documentation says send Base64, the string is valid and, the string is intact throughout the process, but Google will not accept it? I am using the javascript api, I think there is maybe a bug sending the thumbnail using the javascript api. This is the request Request URL:https://www.googleapis.com/upload/drive/v2/files/?uploadType=multipart Request Method:POST Status Code:400 Bad Request **Request Headers** :host:www.googleapis.com :method:POST :path:/upload/drive/v2/files/?uploadType=multipart :scheme:https :version:HTTP/1.1 accept:*/* accept-encoding:gzip,deflate,sdch accept-language:en-US,en;q=0.8 authorization:Bearer ya29.AHES6ZQr...YXDacdY4 content-length:14313 content-type:multipart/mixed; boundary="--mpart_delim" origin:https://www.googleapis.com x-javascript-user-agent:google-api-javascript-client/1.1.0-beta x-origin:https://app.pinteract.com x-referer:https://app.pinteract.com **Query String Parameters** uploadType:multipart **Request Payload** ----mpart_delim Content-Type: application/json {"id":null,"title":"Test Pinup.pint","mimeType":"application/vnd.pinteract.pint","thumbnail":{"mimeType":"image/png","image":"iVBORw0KGgoAAAANSUh...UVORK5CYII%3D"}} ----mpart_delim Content-Type: application/vnd.pinteract.pint Content-Transfer-Encoding: binary { "header" : {"id":"215A660A"} "members" : [ {"id":"100523752012631912873"} ], "manifest" : [ {"id":"0000","ele":[],"own":"100523752012631912873","dtc":1371679680000,"txt":"&Delta; Created","typ":0} ], "elements" : [ {"id":"0F54","x":560,"y":264,"bak":"#44ff44","own":"100523752012631912873","srt":"544","sta":0,"wid":120,"hgt":120,"dtc":0,"rec":"","txt":"This is Note"} ] } ----mpart_delim-- **Response Headers** content-length:10848 content-type:application/json date:Mon, 01 Jul 2013 14:41:33 GMT server:HTTP Upload Server Built on Jun 25 2013 11:32:14 (1372185134) status:400 Bad Request version:HTTP/1.1 This is the Response { "error": { "errors": [ { "domain": "global", "reason": "invalid", "message": "Invalid value for: Not a valid base64 byte string: iVBORw0KGgoAAAANSUhEUg...AASUVORK5CYII%3D" } ], "code": 400, "message": "Invalid value for: Not a valid base64 byte string: iVBORw0KGgoAAAANSU...lRtkAAAAASUVORK5CYII%3D" } } Raw Base64 String iVBORw0KGgoAAAANSUhEUgAAAGAAAABgCAYAAADimHc4AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAADjVJREFUeNrsXX2MHVUVP3fevH27SxfKhyBCsIqoKOhCBBISY5Hyj2JSGuXLP6SJwYAYICgxsUBr1AhqbE2I8Q9jjYE/CLFbP4IxVbaIYjAmS0wUC6ELRGy7tPt2t9vd997MPd6Z+3Xunfsq1r7tvNe56TAfO2/ezO93zu/ce+6ZB0DVqla1qlWtalWrWtWqVrWqVa1qJ1Nj/XCTm5+bHeUcz9+3f25tmvLVSYePIyLIBRhX24yxKXH63Kljw5Ni97UfbVhzpCLgGNs3/jp//r9en72l3UnWCsCvSjrpmQA54Nkq28q3FQn5mucL5OuIsUPi6f5Si6KJVasaT26/+T1vVgS8hfbl3+27br65dE+r1VknrH6IIfkjolmh/A8hwYJv1lySInYXhXc8W49r235++yVPVQQE2lee3n/d3OzS5tZS+2oWurlMYgj4qI45HsApCZYAzgFSsU7FfhSx5+txtHnnHR9+qiJAtPsnD1y4MLf09aXF9q3ZzTAmb8ohQaKtHEDLkGv9qIBHBbgmICXrlKyjKHoirkWbfn3X+EsnLQH37dr3mebs4o8x4WMSfGbAtyQoDSLga8BBeQJqqXE8QQIt12D2syVR2+LcxVpcu/23d1/++ElFwPdeXB555eWZ7x453LozEvuRtvqMgCyAhm7QyI7Sf7BEcI4mCNP9lBKhrD9R2zkJqVwLWfpOYyje8tSXLlsceAIy8Pf8Y98vOq1kXQZ8JG5By06kLZ/5+m8DcEiCMt3XXpCTwN0YkKrYkFu/0CfpCSC3M1IEEeILJ0ca9et/c/fliwNLQAb+SxR8JTkRJUAdy4DOtxCdQGyDb6ALyq0HuEHYyo+OAYlaUkVI7hkAk8ON+Ppd935kceAIMOC3BfjiayUB1gO0FAG4QTjfN9pTjAO0/0+9ICeDBmGyJDQoEzIyjxCr3YKET/7+vitWhIRopQh4+Z/7fyAtX4PPQG/X1JLt19Rit8XfIjDb7uftdWqa0Ox8sNfIF/CvK4mPFAB6yf4uaPzYYit5dKVwWREC7vrV619sLXU+L8Gg4AEBlXgFBdN8JgQ6M+fUHJDJNWiQD1yHkXOlDLLMcz53xTf/dNtASNBXn5l598y++b+L3k2j5gFdU1KkA2/kdEOR9IDsOpSCQBN4M/3nJuCaOEC03pEhJTuJJ0XtbEHsxBG75PmvXb2nrz3gzZnD2wVCjQgYsUAgVm2lRwdiR6LA9w4lS2YNnlcRq/avB653MfrdnueJf/XllP+kryXozl++toF30o9GnoVrvZU9HtINNaD44BFgHSmx4DP/swRkKz9F3WcGCOt9NQ0MwtWXbnn2xr4loL2cbNYDLAdcQgTz9FpbtAaUERANsJEFklHvIOMISiBzrqlJD4w/yKJJEL2lh/qSgDt2vrohTdJL/QdzBlvMtUQ6Io6cXgwBngZWD/xIeZVeM49o5g30KEGh+1Td4g+8f/Mfbuw7Atotbf0EVOchmQuGts5APsh4UNCbSBoDPE/oYt3B48zekzEIVF7Ae+cFPSHgCxOvXpwm/FLHslQ6GZgLfNfFk54iuMwhyAf2aJLTfcHg8cwLLnzwmQ/2DQFCem4pgOR4gzviLQDD3gIRzE1h+PshyaEB3wPYORccImVyUHR1P9s3BCSd9CZ/sOFPsjCV4ymkHViYCJOmYEV5co8xF1DnHrDwfe6cg1wzdZCRE8WY4tN9QYAY9Z4rBj7vzZNpiMWRHqJ5yKMCQcBg3cgJkpFPzjtkRw7x+ruwqxQBkUxNnGgXXbBp8rzSE3BksT3uZDDJUJahD7hKsGFACug+Cx8vegQWRtEsOOzHwO1h8WEQncOiSzpeegLaneQqB2wMEKHWGHhgigUzs2FYsHyAcNzonmfBgneZBfDoORk03nlF6QkQffCL3WdFM4uFSLUWHcuzTqPrTtCdkvSMlwb5ozZVPQHo3g8L+qgjOZ7t5HvHvScUH/ceUMrPrmkTzW+agbRrZswIkXmGKHUbPWCQWDH6shUK8n4g9aVQGQFT3+kYguGdGAoWHOas0nsAqowlnbUyD+BPpoNX56M/R7wAzHkItjLCDaQO1gZXBOcQ3Tego4dzYAsBMBw5yukB8o6ZARiVZSs7l8e8NSOKr60euwRM39J1bZBZG/KsRTuxhRoE+Q6T3gZqNPk0pWUXsfwEaA/IHyQHX1sZM/CiY9OWLH0GJ66ZSwYJuoYgDxMj9QiFeiFbzIWGA3oOBKrvdIWF9j9Z8NUPHoBoQUVq7RJsbnpCTIFNgEeZUANFghmZogWeYVG6bAmirYqgJSwFItC3eNdwqIdwpN7SBx7A0YZdRCo1ECQiMlYvk3WcDJAi7QGBYAzepDyqSXlTpKXWEPAI7sQk7RmWJFPeqGXInNcHHtBKUhiJa/KBsoq0CBSwQJIENg2RP7AqyNIdpYhYnfYI9KYo/SlJWxdKSSHgBTyAQzfZcqc9OfGG0veC4ijaTcsEgdZskqo1W7/jVrbpEhL6N7kPzpwu99dqHljPDaNXqGuvby0cQyQpY+EK8BSsV4i2u/QecLjdmR4biq2Gc5WCZmgSXJwzKeYkqqKyfDqyjZT12VEtFrqZ6FTC0bJ0S7AGMD8PSB2R8RjrOUhK3FPQniIlSpAxXXoCFlrJ1Kp6rLScmVElU2gzf5TFbJIMzawYM8GWBXI4tEoaObrvB5A6UBMLPI8qWj6aLmdeyqjApx6QSqKmjjdePSlL+dQPX5gdjqPVea2OKqqyBVFuXZCukgOg9TvZMVuamGc3/RJFJAMsXqyOowW53CtF0UW6VtK4qpgGaIvtlrhOS6yz8hS5na2xefDha0/vg4FYrsUT4v5vy1PD3JUfSjnKMZvt66PNxWA+k8VtahjdHhCQFzVoebpTC0S2uV+2SOOCkSFl/WitPy9hlN4x0TcTMjOLrZ1GRwMFsno77yVxWjzlWqUTfNH/u1to637GBT8/jxbr6uCOhBR1TqplCKTscNDygzt7gVXPKuM+8ejUbKOWyRCt61TlHqYQihVKSY7+loy0eqZAgkCRrl8d7b8Zk5rXlVSlnC5bF8dasiJOV8Zp6RHb0DzUA/npmQfkwXi5s80CwF2rRbdc0PEQ9eJEwdqJ9yQFj+HBc5MA+Nx5m9LuJ0R+8hhhJCgnd1uvcOoZAS/sn9sqBmVNCnoIzAJwafdj+uUKnvKuwCeBgCutHozM+DEhUYDbNSgScu1vit2tfUfA/CPrmgvtRHqBM4gCByBOQKTAUPAl8FwAb4tpE97dS5JuZFNZUtadKsATQ4RYAIhHwLameJa+IyBrfzswv3WxI7yASIEPjvumilpS1xMK3UjHe3xC1DXQexmDyhEJwh2OLviuJwjgcWsvMep5efr4t55b//ZThnbk4wBw+/6F2n3wS0v8EnUyIYM0tex1Q8l26qUkUjViThT4HZSLDLyQB9+ODLzZuTcI65/oawKyduW3/7zj9OH6+ihEgDcA82t3GEBxMkQBqntFvPC+MDjdS5qm0IGWgt/RwKveT7Yvjk8I8G/oNTbxShDw4qHDGz901tia0bg2jirvo9fcIQC7lJ+gk6RHPdEPUMhcOj9ZAMW353O5kgAroNU2pxKUpxw2rgQ2K/aS3jsf3D1+/qrhp0fE2MCkIzwJ6lYy6Fu/JcCf8QqMB8Baf6JAT8i2lp9EyY441hSfuWbukXVTA0VATsIDu8fPO6Xx9HBGAi1NJ2XkAMUKtcKcJ523JeBz5ycLbMo5VUDbtZYgua/BF72fpvjMNc0VAn/FCcjaBQ9Mjr9jtJF7An1zhQZiO0VvM6VQSEdbDwCPACpDKenVpMbybW9Hb2ejXfHpFQX/hBCQk7Bpcvzs4aEdo3G0hr4fHEGoiNdNRdOUhEuElSOdTEvABt2EkNFx+v3CCwCmUfZ4plYaixP2Yx2n3r9r9QWnNHasrsdrQ68KwX8hAOiEjJ421H18oGkFRQSXg6+O7gUZr4BJcYUbejnYKiUBuq3ZNHnPGY36Q42IrWYBL2B+4Szt/2vZAfJDTd6kigZeSw7J82R6v2X2kXVbT+Tzl+IHm4Q3rDlnuP6Q6KbeVjfxAPWbis7NojcdqSfWubJ8riUIrAfo1ENiPAS2i2NbRE9n+kQ/e6l+suw0QcSZwhtEL8kQQUvbTb0aLec0lk9Gv2AHYwkhQvzbLpYthx6+drosz1zKH+3LPGIsrq2PDu79/vA5F8p3igMEAFjZQV+CrCxNicM/FdY/UQaL7wsCdBsZGcH6GefB2EVXwsi574Pa294FSeM0aNVPpeMyFYQRhtrzEC0dgnTmFWi9sQcW9k5NLr2x5+PQm7ra/klF/F8WcngG4r1/hLG5F+HsQ+fA6OgoDDUaEMdx9rtv0vLTFNqdDrSWl2FhYQEOHNgPyeysCLOzzAsfpWsRDH5jZfb2gSQA3fQFKzMJJ4sHlJaEeFA9QHlB6G1YLFNMGGAPQPqMrKyeEMNgN5pg5V08AU6kNwwsASQQUy/nXdzkhElSPODgs4DMhkioPKA3RJgfvwoFCN6FkIqA4xwDQgTo90eSKgj3oHGe0q5oHBgVZwR0ql5Qz2MAhjwg+2O7Goj1VvtpD6jugb8AJUrMDagEcToSjgn48yCrzkvTBjYZx7khoK6e82AZgu5JQ4Bqi+oZ/10m3e8bAgSQE8dOQE7CJMh3e5fL+oylJiCKom3HSoBYDqdp+pjYLfX/Ta/UBBw5cmSSMbb1WAhgLNqYJMnLZZfL0seAZrN5rwDzfyGhWa/XNx48+OaT/RCv+iIIT0/vvbfRaFwjvOFoMSErLdxeq9Uu27v3le390mFg0IftpptvXetURXDefPyxn01B1apWtapVrWpVq1rVqla1t9L+I8AAvydNUrElRtkAAAAASUVORK5CYII="}],"code":400,"message":"Invalidvaluefor:Notavalidbase64bytestring:iVBORw0KGgoAAAANSUhEUgAAAGAAAABgCAYAAADimHc4AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAADjVJREFUeNrsXX2MHVUVP3fevH27SxfKhyBCsIqoKOhCBBISY5Hyj2JSGuXLP6SJwYAYICgxsUBr1AhqbE2I8Q9jjYE/CLFbP4IxVbaIYjAmS0wUC6ELRGy7tPt2t9vd997MPd6Z+3Xunfsq1r7tvNe56TAfO2/ezO93zu/ce+6ZB0DVqla1qlWtalWrWtWqVrWqVa1qJ1Nj/XCTm5+bHeUcz9+3f25tmvLVSYePIyLIBRhX24yxKXH63Kljw5Ni97UfbVhzpCLgGNs3/jp//r9en72l3UnWCsCvSjrpmQA54Nkq28q3FQn5mucL5OuIsUPi6f5Si6KJVasaT26/+T1vVgS8hfbl3+27br65dE+r1VknrH6IIfkjolmh/A8hwYJv1lySInYXhXc8W49r235++yVPVQQE2lee3n/d3OzS5tZS+2oWurlMYgj4qI45HsApCZYAzgFSsU7FfhSx5+txtHnnHR9+qiJAtPsnD1y4MLf09aXF9q3ZzTAmb8ohQaKtHEDLkGv9qIBHBbgmICXrlKyjKHoirkWbfn3X+EsnLQH37dr3mebs4o8x4WMSfGbAtyQoDSLga8BBeQJqqXE8QQIt12D2syVR2+LcxVpcu/23d1/++ElFwPdeXB555eWZ7x453LozEvuRtvqMgCyAhm7QyI7Sf7BEcI4mCNP9lBKhrD9R2zkJqVwLWfpOYyje8tSXLlsceAIy8Pf8Y98vOq1kXQZ8JG5By06kLZ/5+m8DcEiCMt3XXpCTwN0YkKrYkFu/0CfpCSC3M1IEEeILJ0ca9et/c/fliwNLQAb+SxR8JTkRJUAdy4DOtxCdQGyDb6ALyq0HuEHYyo+OAYlaUkVI7hkAk8ON+Ppd935kceAIMOC3BfjiayUB1gO0FAG4QTjfN9pTjAO0/0+9ICeDBmGyJDQoEzIyjxCr3YKET/7+vitWhIRopQh4+Z/7fyAtX4PPQG/X1JLt19Rit8XfIjDb7uftdWqa0Ox8sNfIF/CvK4mPFAB6yf4uaPzYYit5dKVwWREC7vrV619sLXU+L8Gg4AEBlXgFBdN8JgQ6M+fUHJDJNWiQD1yHkXOlDLLMcz53xTf/dNtASNBXn5l598y++b+L3k2j5gFdU1KkA2/kdEOR9IDsOpSCQBN4M/3nJuCaOEC03pEhJTuJJ0XtbEHsxBG75PmvXb2nrz3gzZnD2wVCjQgYsUAgVm2lRwdiR6LA9w4lS2YNnlcRq/avB653MfrdnueJf/XllP+kryXozl++toF30o9GnoVrvZU9HtINNaD44BFgHSmx4DP/swRkKz9F3WcGCOt9NQ0MwtWXbnn2xr4loL2cbNYDLAdcQgTz9FpbtAaUERANsJEFklHvIOMISiBzrqlJD4w/yKJJEL2lh/qSgDt2vrohTdJL/QdzBlvMtUQ6Io6cXgwBngZWD/xIeZVeM49o5g30KEGh+1Td4g+8f/Mfbuw7Atotbf0EVOchmQuGts5APsh4UNCbSBoDPE/oYt3B48zekzEIVF7Ae+cFPSHgCxOvXpwm/FLHslQ6GZgLfNfFk54iuMwhyAf2aJLTfcHg8cwLLnzwmQ/2DQFCem4pgOR4gzviLQDD3gIRzE1h+PshyaEB3wPYORccImVyUHR1P9s3BCSd9CZ/sOFPsjCV4ymkHViYCJOmYEV5co8xF1DnHrDwfe6cg1wzdZCRE8WY4tN9QYAY9Z4rBj7vzZNpiMWRHqJ5yKMCQcBg3cgJkpFPzjtkRw7x+ruwqxQBkUxNnGgXXbBp8rzSE3BksT3uZDDJUJahD7hKsGFACug+Cx8vegQWRtEsOOzHwO1h8WEQncOiSzpeegLaneQqB2wMEKHWGHhgigUzs2FYsHyAcNzonmfBgneZBfDoORk03nlF6QkQffCL3WdFM4uFSLUWHcuzTqPrTtCdkvSMlwb5ozZVPQHo3g8L+qgjOZ7t5HvHvScUH/ceUMrPrmkTzW+agbRrZswIkXmGKHUbPWCQWDH6shUK8n4g9aVQGQFT3+kYguGdGAoWHOas0nsAqowlnbUyD+BPpoNX56M/R7wAzHkItjLCDaQO1gZXBOcQ3Tego4dzYAsBMBw5yukB8o6ZARiVZSs7l8e8NSOKr60euwRM39J1bZBZG/KsRTuxhRoE+Q6T3gZqNPk0pWUXsfwEaA/IHyQHX1sZM/CiY9OWLH0GJ66ZSwYJuoYgDxMj9QiFeiFbzIWGA3oOBKrvdIWF9j9Z8NUPHoBoQUVq7RJsbnpCTIFNgEeZUANFghmZogWeYVG6bAmirYqgJSwFItC3eNdwqIdwpN7SBx7A0YZdRCo1ECQiMlYvk3WcDJAi7QGBYAzepDyqSXlTpKXWEPAI7sQk7RmWJFPeqGXInNcHHtBKUhiJa/KBsoq0CBSwQJIENg2RP7AqyNIdpYhYnfYI9KYo/SlJWxdKSSHgBTyAQzfZcqc9OfGG0veC4ijaTcsEgdZskqo1W7/jVrbpEhL6N7kPzpwu99dqHljPDaNXqGuvby0cQyQpY+EK8BSsV4i2u/QecLjdmR4biq2Gc5WCZmgSXJwzKeYkqqKyfDqyjZT12VEtFrqZ6FTC0bJ0S7AGMD8PSB2R8RjrOUhK3FPQniIlSpAxXXoCFlrJ1Kp6rLScmVElU2gzf5TFbJIMzawYM8GWBXI4tEoaObrvB5A6UBMLPI8qWj6aLmdeyqjApx6QSqKmjjdePSlL+dQPX5gdjqPVea2OKqqyBVFuXZCukgOg9TvZMVuamGc3/RJFJAMsXqyOowW53CtF0UW6VtK4qpgGaIvtlrhOS6yz8hS5na2xefDha0/vg4FYrsUT4v5vy1PD3JUfSjnKMZvt66PNxWA+k8VtahjdHhCQFzVoebpTC0S2uV+2SOOCkSFl/WitPy9hlN4x0TcTMjOLrZ1GRwMFsno77yVxWjzlWqUTfNH/u1to637GBT8/jxbr6uCOhBR1TqplCKTscNDygzt7gVXPKuM+8ejUbKOWyRCt61TlHqYQihVKSY7+loy0eqZAgkCRrl8d7b8Zk5rXlVSlnC5bF8dasiJOV8Zp6RHb0DzUA/npmQfkwXi5s80CwF2rRbdc0PEQ9eJEwdqJ9yQFj+HBc5MA+Nx5m9LuJ0R+8hhhJCgnd1uvcOoZAS/sn9sqBmVNCnoIzAJwafdj+uUKnvKuwCeBgCutHozM+DEhUYDbNSgScu1vit2tfUfA/CPrmgvtRHqBM4gCByBOQKTAUPAl8FwAb4tpE97dS5JuZFNZUtadKsATQ4RYAIhHwLameJa+IyBrfzswv3WxI7yASIEPjvumilpS1xMK3UjHe3xC1DXQexmDyhEJwh2OLviuJwjgcWsvMep5efr4t55b//ZThnbk4wBw+/6F2n3wS0v8EnUyIYM0tex1Q8l26qUkUjViThT4HZSLDLyQB9+ODLzZuTcI65/oawKyduW3/7zj9OH6+ihEgDcA82t3GEBxMkQBqntFvPC+MDjdS5qm0IGWgt/RwKveT7Yvjk8I8G/oNTbxShDw4qHDGz901tia0bg2jirvo9fcIQC7lJ+gk6RHPdEPUMhcOj9ZAMW353O5kgAroNU2pxKUpxw2rgQ2K/aS3jsf3D1+/qrhp0fE2MCkIzwJ6lYy6Fu/JcCf8QqMB8Baf6JAT8i2lp9EyY441hSfuWbukXVTA0VATsIDu8fPO6Xx9HBGAi1NJ2XkAMUKtcKcJ523JeBz5ycLbMo5VUDbtZYgua/BF72fpvjMNc0VAn/FCcjaBQ9Mjr9jtJF7An1zhQZiO0VvM6VQSEdbDwCPACpDKenVpMbybW9Hb2ejXfHpFQX/hBCQk7Bpcvzs4aEdo3G0hr4fHEGoiNdNRdOUhEuElSOdTEvABt2EkNFx+v3CCwCmUfZ4plYaixP2Yx2n3r9r9QWnNHasrsdrQ68KwX8hAOiEjJ421H18oGkFRQSXg6+O7gUZr4BJcYUbejnYKiUBuq3ZNHnPGY36Q42IrWYBL2B+4Szt/2vZAfJDTd6kigZeSw7J82R6v2X2kXVbT+Tzl+IHm4Q3rDlnuP6Q6KbeVjfxAPWbis7NojcdqSfWubJ8riUIrAfo1ENiPAS2i2NbRE9n+kQ/e6l+suw0QcSZwhtEL8kQQUvbTb0aLec0lk9Gv2AHYwkhQvzbLpYthx6+drosz1zKH+3LPGIsrq2PDu79/vA5F8p3igMEAFjZQV+CrCxNicM/FdY/UQaL7wsCdBsZGcH6GefB2EVXwsi574Pa294FSeM0aNVPpeMyFYQRhtrzEC0dgnTmFWi9sQcW9k5NLr2x5+PQm7ra/klF/F8WcngG4r1/hLG5F+HsQ+fA6OgoDDUaEMdx9rtv0vLTFNqdDrSWl2FhYQEOHNgPyeysCLOzzAsfpWsRDH5jZfb2gSQA3fQFKzMJJ4sHlJaEeFA9QHlB6G1YLFNMGGAPQPqMrKyeEMNgN5pg5V08AU6kNwwsASQQUy/nXdzkhElSPODgs4DMhkioPKA3RJgfvwoFCN6FkIqA4xwDQgTo90eSKgj3oHGe0q5oHBgVZwR0ql5Qz2MAhjwg+2O7Goj1VvtpD6jugb8AJUrMDagEcToSjgn48yCrzkvTBjYZx7khoK6e82AZgu5JQ4Bqi+oZ/10m3e8bAgSQE8dOQE7CJMh3e5fL+oylJiCKom3HSoBYDqdp+pjYLfX/Ta/UBBw5cmSSMbb1WAhgLNqYJMnLZZfL0seAZrN5rwDzfyGhWa/XNx48+OaT/RCv+iIIT0/vvbfRaFwjvOFoMSErLdxeq9Uu27v3le390mFg0IftpptvXetURXDefPyxn01B1apWtapVrWpVq1rVqla1t9L+I8AAvydNUrElRtkAAAAASUVORK5CYII= Url Encoded Base64 String iVBORw0KGgoAAAANSUhEUgAAAGAAAABgCAYAAADimHc4AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAADjVJREFUeNrsXX2MHVUVP3fevH27SxfKhyBCsIqoKOhCBBISY5Hyj2JSGuXLP6SJwYAYICgxsUBr1AhqbE2I8Q9jjYE%2FCLFbP4IxVbaIYjAmS0wUC6ELRGy7tPt2t9vd997MPd6Z%2B3Xunfsq1r7tvNe56TAfO2%2FezO93zu%2Fce%2B6ZB0DVqla1qlWtalWrWtWqVrWqVa1qJ1Nj%2FXCTm5%2BbHeUcz9%2B3f25tmvLVSYePIyLIBRhX24yxKXH63Kljw5Ni97UfbVhzpCLgGNs3%2Fjp%2F%2Fr9en72l3UnWCsCvSjrpmQA54Nkq28q3FQn5mucL5OuIsUPi6f5Si6KJVasaT26%2F%2BT1vVgS8hfbl3%2B27br65dE%2Br1VknrH6IIfkjolmh%2FA8hwYJv1lySInYXhXc8W49r235%2B%2ByVPVQQE2lee3n%2Fd3OzS5tZS%2B2oWurlMYgj4qI45HsApCZYAzgFSsU7FfhSx5%2BtxtHnnHR9%2BqiJAtPsnD1y4MLf09aXF9q3ZzTAmb8ohQaKtHEDLkGv9qIBHBbgmICXrlKyjKHoirkWbfn3X%2BEsnLQH37dr3mebs4o8x4WMSfGbAtyQoDSLga8BBeQJqqXE8QQIt12D2syVR2%2BLcxVpcu%2F23d1%2F%2B%2BElFwPdeXB555eWZ7x453LozEvuRtvqMgCyAhm7QyI7Sf7BEcI4mCNP9lBKhrD9R2zkJqVwLWfpOYyje8tSXLlsceAIy8Pf8Y98vOq1kXQZ8JG5By06kLZ%2F5%2Bm8DcEiCMt3XXpCTwN0YkKrYkFu%2F0CfpCSC3M1IEEeILJ0ca9et%2Fc%2FfliwNLQAb%2BSxR8JTkRJUAdy4DOtxCdQGyDb6ALyq0HuEHYyo%2BOAYlaUkVI7hkAk8ON%2BPpd935kceAIMOC3BfjiayUB1gO0FAG4QTjfN9pTjAO0%2F0%2B9ICeDBmGyJDQoEzIyjxCr3YKET%2F7%2BvitWhIRopQh4%2BZ%2F7fyAtX4PPQG%2FX1JLt19Rit8XfIjDb7uftdWqa0Ox8sNfIF%2FCvK4mPFAB6yf4uaPzYYit5dKVwWREC7vrV619sLXU%2BL8Gg4AEBlXgFBdN8JgQ6M%2BfUHJDJNWiQD1yHkXOlDLLMcz53xTf%2FdNtASNBXn5l598y%2B%2Bb%2BL3k2j5gFdU1KkA2%2FkdEOR9IDsOpSCQBN4M%2F3nJuCaOEC03pEhJTuJJ0XtbEHsxBG75PmvXb2nrz3gzZnD2wVCjQgYsUAgVm2lRwdiR6LA9w4lS2YNnlcRq%2FavB653MfrdnueJf%2FXllP%2BkryXozl%2B%2BtoF30o9GnoVrvZU9HtINNaD44BFgHSmx4DP%2FswRkKz9F3WcGCOt9NQ0MwtWXbnn2xr4loL2cbNYDLAdcQgTz9FpbtAaUERANsJEFklHvIOMISiBzrqlJD4w%2FyKJJEL2lh%2FqSgDt2vrohTdJL%2FQdzBlvMtUQ6Io6cXgwBngZWD%2FxIeZVeM49o5g30KEGh%2B1Td4g%2B8f%2FMfbuw7Atotbf0EVOchmQuGts5APsh4UNCbSBoDPE%2FoYt3B48zekzEIVF7Ae%2BcFPSHgCxOvXpwm%2FFLHslQ6GZgLfNfFk54iuMwhyAf2aJLTfcHg8cwLLnzwmQ%2F2DQFCem4pgOR4gzviLQDD3gIRzE1h%2BPshyaEB3wPYORccImVyUHR1P9s3BCSd9CZ%2FsOFPsjCV4ymkHViYCJOmYEV5co8xF1DnHrDwfe6cg1wzdZCRE8WY4tN9QYAY9Z4rBj7vzZNpiMWRHqJ5yKMCQcBg3cgJkpFPzjtkRw7x%2BruwqxQBkUxNnGgXXbBp8rzSE3BksT3uZDDJUJahD7hKsGFACug%2BCx8vegQWRtEsOOzHwO1h8WEQncOiSzpeegLaneQqB2wMEKHWGHhgigUzs2FYsHyAcNzonmfBgneZBfDoORk03nlF6QkQffCL3WdFM4uFSLUWHcuzTqPrTtCdkvSMlwb5ozZVPQHo3g8L%2BqgjOZ7t5HvHvScUH%2FceUMrPrmkTzW%2BagbRrZswIkXmGKHUbPWCQWDH6shUK8n4g9aVQGQFT3%2BkYguGdGAoWHOas0nsAqowlnbUyD%2BBPpoNX56M%2FR7wAzHkItjLCDaQO1gZXBOcQ3Tego4dzYAsBMBw5yukB8o6ZARiVZSs7l8e8NSOKr60euwRM39J1bZBZG%2FKsRTuxhRoE%2BQ6T3gZqNPk0pWUXsfwEaA%2FIHyQHX1sZM%2FCiY9OWLH0GJ66ZSwYJuoYgDxMj9QiFeiFbzIWGA3oOBKrvdIWF9j9Z8NUPHoBoQUVq7RJsbnpCTIFNgEeZUANFghmZogWeYVG6bAmirYqgJSwFItC3eNdwqIdwpN7SBx7A0YZdRCo1ECQiMlYvk3WcDJAi7QGBYAzepDyqSXlTpKXWEPAI7sQk7RmWJFPeqGXInNcHHtBKUhiJa%2FKBsoq0CBSwQJIENg2RP7AqyNIdpYhYnfYI9KYo%2FSlJWxdKSSHgBTyAQzfZcqc9OfGG0veC4ijaTcsEgdZskqo1W7%2FjVrbpEhL6N7kPzpwu99dqHljPDaNXqGuvby0cQyQpY%2BEK8BSsV4i2u%2FQecLjdmR4biq2Gc5WCZmgSXJwzKeYkqqKyfDqyjZT12VEtFrqZ6FTC0bJ0S7AGMD8PSB2R8RjrOUhK3FPQniIlSpAxXXoCFlrJ1Kp6rLScmVElU2gzf5TFbJIMzawYM8GWBXI4tEoaObrvB5A6UBMLPI8qWj6aLmdeyqjApx6QSqKmjjdePSlL%2BdQPX5gdjqPVea2OKqqyBVFuXZCukgOg9TvZMVuamGc3%2FRJFJAMsXqyOowW53CtF0UW6VtK4qpgGaIvtlrhOS6yz8hS5na2xefDha0%2Fvg4FYrsUT4v5vy1PD3JUfSjnKMZvt66PNxWA%2Bk8VtahjdHhCQFzVoebpTC0S2uV%2B2SOOCkSFl%2FWitPy9hlN4x0TcTMjOLrZ1GRwMFsno77yVxWjzlWqUTfNH%2Fu1to637GBT8%2Fjxbr6uCOhBR1TqplCKTscNDygzt7gVXPKuM%2B8ejUbKOWyRCt61TlHqYQihVKSY7%2Bloy0eqZAgkCRrl8d7b8Zk5rXlVSlnC5bF8dasiJOV8Zp6RHb0DzUA%2FnpmQfkwXi5s80CwF2rRbdc0PEQ9eJEwdqJ9yQFj%2BHBc5MA%2BNx5m9LuJ0R%2B8hhhJCgnd1uvcOoZAS%2Fsn9sqBmVNCnoIzAJwafdj%2BuUKnvKuwCeBgCutHozM%2BDEhUYDbNSgScu1vit2tfUfA%2FCPrmgvtRHqBM4gCByBOQKTAUPAl8FwAb4tpE97dS5JuZFNZUtadKsATQ4RYAIhHwLameJa%2BIyBrfzswv3WxI7yASIEPjvumilpS1xMK3UjHe3xC1DXQexmDyhEJwh2OLviuJwjgcWsvMep5efr4t55b%2F%2FZThnbk4wBw%2B%2F6F2n3wS0v8EnUyIYM0tex1Q8l26qUkUjViThT4HZSLDLyQB9%2BODLzZuTcI65%2FoawKyduW3%2F7zj9OH6%2BihEgDcA82t3GEBxMkQBqntFvPC%2BMDjdS5qm0IGWgt%2FRwKveT7Yvjk8I8G%2FoNTbxShDw4qHDGz901tia0bg2jirvo9fcIQC7lJ%2Bgk6RHPdEPUMhcOj9ZAMW353O5kgAroNU2pxKUpxw2rgQ2K%2FaS3jsf3D1%2B%2Fqrhp0fE2MCkIzwJ6lYy6Fu%2FJcCf8QqMB8Baf6JAT8i2lp9EyY441hSfuWbukXVTA0VATsIDu8fPO6Xx9HBGAi1NJ2XkAMUKtcKcJ523JeBz5ycLbMo5VUDbtZYgua%2FBF72fpvjMNc0VAn%2FFCcjaBQ9Mjr9jtJF7An1zhQZiO0VvM6VQSEdbDwCPACpDKenVpMbybW9Hb2ejXfHpFQX%2FhBCQk7Bpcvzs4aEdo3G0hr4fHEGoiNdNRdOUhEuElSOdTEvABt2EkNFx%2Bv3CCwCmUfZ4plYaixP2Yx2n3r9r9QWnNHasrsdrQ68KwX8hAOiEjJ421H18oGkFRQSXg6%2BO7gUZr4BJcYUbejnYKiUBuq3ZNHnPGY36Q42IrWYBL2B%2B4Szt%2F2vZAfJDTd6kigZeSw7J82R6v2X2kXVbT%2BTzl%2BIHm4Q3rDlnuP6Q6KbeVjfxAPWbis7NojcdqSfWubJ8riUIrAfo1ENiPAS2i2NbRE9n%2BkQ%2Fe6l%2Bsuw0QcSZwhtEL8kQQUvbTb0aLec0lk9Gv2AHYwkhQvzbLpYthx6%2Bdrosz1zKH%2B3LPGIsrq2PDu79%2FvA5F8p3igMEAFjZQV%2BCrCxNicM%2FFdY%2FUQaL7wsCdBsZGcH6GefB2EVXwsi574Pa294FSeM0aNVPpeMyFYQRhtrzEC0dgnTmFWi9sQcW9k5NLr2x5%2BPQm7ra%2FklF%2FF8WcngG4r1%2FhLG5F%2BHsQ%2BfA6OgoDDUaEMdx9rtv0vLTFNqdDrSWl2FhYQEOHNgPyeysCLOzzAsfpWsRDH5jZfb2gSQA3fQFKzMJJ4sHlJaEeFA9QHlB6G1YLFNMGGAPQPqMrKyeEMNgN5pg5V08AU6kNwwsASQQUy%2FnXdzkhElSPODgs4DMhkioPKA3RJgfvwoFCN6FkIqA4xwDQgTo90eSKgj3oHGe0q5oHBgVZwR0ql5Qz2MAhjwg%2B2O7Goj1VvtpD6jugb8AJUrMDagEcToSjgn48yCrzkvTBjYZx7khoK6e82AZgu5JQ4Bqi%2BoZ%2F10m3e8bAgSQE8dOQE7CJMh3e5fL%2BoylJiCKom3HSoBYDqdp%2BpjYLfX%2FTa%2FUBBw5cmSSMbb1WAhgLNqYJMnLZZfL0seAZrN5rwDzfyGhWa%2FXNx48%2BOaT%2FRCv%2BiIIT0%2FvvbfRaFwjvOFoMSErLdxeq9Uu27v3le390mFg0IftpptvXetURXDefPyxn01B1apWtapVrWpVq1rVqla1t9L%2BI8AAvydNUrElRtkAAAAASUVORK5CYII%3D

  Read the article

• Django: TypeError: 'str' object is not callable, referer: http://xxx

  - by user705415

  I've been wondering why when I set the settings.py of my django project 'arvindemo' debug = Flase and deploy it on Apache with mod_wsgi, I got the 500 Internal Server Error. Env: Django 1.4.0 Python 2.7.2 mod_wsgi 2.8 OS centOS Here is the recap: Visit the homepage, go to sub page A/B/C/D, and fill some forms, then submit it to the Apache server. Once click 'submit' button, I will get the '500 Internal Server Error', and the error_log listed below(Traceback): [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] Traceback (most recent call last): [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] File "/opt/python2.7/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 241, in __call__ [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] response = self.get_response(request) [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] File "/opt/python2.7/lib/python2.7/site-packages/django/core/handlers/base.py", line 179, in get_response [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] response = self.handle_uncaught_exception(request, resolver, sys.exc_info()) [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] File "/opt/python2.7/lib/python2.7/site-packages/django/core/handlers/base.py", line 224, in handle_uncaught_exception [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] if resolver.urlconf_module is None: [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] File "/opt/python2.7/lib/python2.7/site-packages/django/core/urlresolvers.py", line 323, in urlconf_module [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] self._urlconf_module = import_module(self.urlconf_name) [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] File "/opt/python2.7/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] __import__(name) [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] File "/opt/web/django/arvindemo/arvindemo/../arvindemo/urls.py", line 23, in <module> [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] url(r'^submitPage$', name=submitPage), [Tue Apr 10 10:07:20 2012] [error] [client 122.198.133.250] TypeError: url() takes at least 2 arguments (2 given) When using django runserver, I set arvindemo.settings debug = True, everything is OK. But things changed once I set debug = Flase. Here is my views.py from django.http import HttpResponseRedirect from django.http import HttpResponse, HttpResponseServerError from django.shortcuts import render_to_response import datetime, string from user_info.models import * from django.template import Context, loader, RequestContext import settings def hello(request): return HttpResponse("hello girl") def helpPage(request): return render_to_response('kktHelp.html') def server_error(request, template_name='500.html'): return render_to_response(template_name, context_instance = RequestContext(request) ) def page404(request): return render_to_response('404.html') def submitPage(request): post = request.POST Mall = 'goodsName' Contest = 'ojs' Presentation = 'addr' WeatherReport = 'city' Habit = 'task' if Mall in post: return submitMall(request) elif Contest in post: return submitContest(request) elif Presentation in post: return submitPresentation(request) elif Habit in post: return submitHabit(request) elif WeatherReport in post: return submitWeather(request) else: return HttpResponse(request.POST) return HttpResponseRedirect('404') def submitXXX(): ..... def xxxx(): .... Here comes the urls.py from django.conf.urls import patterns, include, url from views import * from django.conf import settings handler500 = 'server_error' urlpatterns = patterns('', url(r'^hello/$', hello), # hello world url(r'^$', homePage), url(r'^time/$', getTime), url(r'^time/plus/(\d{1,2})/$', hoursAhead), url(r'^Ttime/$', templateGetTime), url(r'^Mall$', templateMall), url(r'^Contest$', templateContest), url(r'^Presentation$', templatePresentation), url(r'^Habit$', templateHabit), url(r'^Weather$', templateWeather), url(r'^Help$', helpPage), url(r'^404$', page404), url(r'^500$', server_error), url(r'^submitPage$', submitPage), url(r'^submitMall$', submitMall), url(r'^submitContest$', submitContest), url(r'^submitPresentation$', submitPresentation), url(r'^submitHabit$', submitHabit), url(r'^submitWeather$', submitWeather), url(r'^terms$', terms), url(r'^privacy$', privacy), url(r'^thanks$', thanks), url(r'^about$', about), url(r'^static/(?P<path>.*)$','django.views.static.serve',{'document_root':settings.STATICFILES_DIRS}), ) I'm sure there is no syntax error in my django project,cause when I use django runserver, everything is fine. Anyone can help ? Best regards

  Read the article

• Code golf: Word frequency chart

  - by ChristopheD

  The challenge: Build an ASCII chart of the most commonly used words in a given text. The rules: Only accept a-z and A-Z (alphabetic characters) as part of a word. Ignore casing (She == she for our purpose). Ignore the following words (quite arbitary, I know): the, and, of, to, a, i, it, in, or, is Clarification: considering don't: this would be taken as 2 different 'words' in the ranges a-z and A-Z: (don and t). Optionally (it's too late to be formally changing the specifications now) you may choose to drop all single-letter 'words' (this could potentially make for a shortening of the ignore list too). Parse a given text (read a file specified via command line arguments or piped in; presume us-ascii) and build us a word frequency chart with the following characteristics: Display the chart (also see the example below) for the 22 most common words (ordered by descending frequency). The bar width represents the number of occurences (frequency) of the word (proportionally). Append one space and print the word. Make sure these bars (plus space-word-space) always fit: bar + [space] + word + [space] should be always <= 80 characters (make sure you account for possible differing bar and word lenghts: e.g.: the second most common word could be a lot longer then the first while not differing so much in frequency). Maximize bar width within these constraints and scale the bars appropriately (according to the frequencies they represent). An example: The text for the example can be found here (Alice's Adventures in Wonderland, by Lewis Carroll). This specific text would yield the following chart: _________________________________________________________________________ |_________________________________________________________________________| she |_______________________________________________________________| you |____________________________________________________________| said |____________________________________________________| alice |______________________________________________| was |__________________________________________| that |___________________________________| as |_______________________________| her |____________________________| with |____________________________| at |___________________________| s |___________________________| t |_________________________| on |_________________________| all |______________________| this |______________________| for |______________________| had |_____________________| but |____________________| be |____________________| not |___________________| they |__________________| so For your information: these are the frequencies the above chart is built upon: [('she', 553), ('you', 481), ('said', 462), ('alice', 403), ('was', 358), ('that ', 330), ('as', 274), ('her', 248), ('with', 227), ('at', 227), ('s', 219), ('t' , 218), ('on', 204), ('all', 200), ('this', 181), ('for', 179), ('had', 178), (' but', 175), ('be', 167), ('not', 166), ('they', 155), ('so', 152)] A second example (to check if you implemented the complete spec): Replace every occurence of you in the linked Alice in Wonderland file with superlongstringstring: ________________________________________________________________ |________________________________________________________________| she |_______________________________________________________| superlongstringstring |_____________________________________________________| said |______________________________________________| alice |________________________________________| was |_____________________________________| that |______________________________| as |___________________________| her |_________________________| with |_________________________| at |________________________| s |________________________| t |______________________| on |_____________________| all |___________________| this |___________________| for |___________________| had |__________________| but |_________________| be |_________________| not |________________| they |________________| so The winner: Shortest solution (by character count, per language). Have fun! Edit: Table summarizing the results so far (2012-02-15) (originally added by user Nas Banov): Language Relaxed Strict ========= ======= ====== GolfScript 130 143 Perl 185 Windows PowerShell 148 199 Mathematica 199 Ruby 185 205 Unix Toolchain 194 228 Python 183 243 Clojure 282 Scala 311 Haskell 333 Awk 336 R 298 Javascript 304 354 Groovy 321 Matlab 404 C# 422 Smalltalk 386 PHP 450 F# 452 TSQL 483 507 The numbers represent the length of the shortest solution in a specific language. "Strict" refers to a solution that implements the spec completely (draws |____| bars, closes the first bar on top with a ____ line, accounts for the possibility of long words with high frequency etc). "Relaxed" means some liberties were taken to shorten to solution. Only solutions shorter then 500 characters are included. The list of languages is sorted by the length of the 'strict' solution. 'Unix Toolchain' is used to signify various solutions that use traditional *nix shell plus a mix of tools (like grep, tr, sort, uniq, head, perl, awk).

  Read the article

• Error when opening .tar.gz via Shell to install Apache Maven

  - by adamsquared

  Thank you in advance for the help. My Goal: To install apache maven per its websites instructions (http://maven.apache.org/download.html), in order to install the JUNG package according to its install instructions (http://sourceforge.net/apps/trac/jung/wiki/JUNGManual), so I can use the JUNG classes in various Java GUIs. The Problem: I get an error message when I try to extract the apache-maven .gz (install?) file in shell. Background: I'm trying to install the JUNG (http://jung.sourceforge.net/index.html) package to my system's Java, so I can write object-oriented code using various GUIs (Ecliplse, Dr. Java) using the classes in JUNG. I don't understand how the building/installing process works, and how I can get what I build/install to work on various GUIs and the command line. I'm new to shell and the command line, and mostly have experience using a simple IDE (DrJava, Python IDLE, R GUI) to write and compile object-oriented code. Machine: Mac OSX 10.5.8 32-bit. The Instructions: For the maven building Extract the distribution archive, i.e. apache-maven-3.0.4-bin.tar.gz to the directory you wish to install Maven 3.0.4. These instructions assume you chose /usr/local/apache-maven. The subdirectory apache-maven-3.0.4 will be created from the archive. ... for the JUNG installation Appendix: How to Build JUNG This is a brief intro to building JUNG jars with maven2 (the build system that JUNG currently uses). First, ensure that you have a JDK of at least version 1.5: JUNG 2.0+ requires Java 1.5+. Ensure that your JAVA_HOME variable is set to the location of the JDK. On a Windows platform, you may have a separate JRE (Java Runtime Environment) and JDK (Java Development Kit). The JRE has no capability to compile Java source files, so you must have a JDK installed. If your JAVA_HOME variable is set to the location of the JRE, and not the location of the JDK, you will be unable to compile. Get Maven Download and install maven2 from maven.apache.org: http://maven.apache.org/download.html At time of writing (early December 2009), the latest version was maven-2.2.1. Install the downloaded maven2 (there are installation instructions on the Maven website). Follow the installation instructions and confirm a successful installation by typing 'mvn --version' in a command terminal window. Get JUNG ... What I Did: I downloaded the file apache-maven-2.2.1-bin.tar.gz. The JUNG website specified to use apache maven 2. I wanted to stick to the recommended installation instructions, but I couldn't get to /usr on my GUI (i've noticed you click on the MacHD symbol on the desktop its missing several directories/folders that you can see using the shell using the ls command at root directory I couldn't find a way to access the file using my mac GUI. Therefore, I used the shell to navigate to the root directory and then to /usr/local, and used the mkdir command to make the directory apache-maven and entered it. I then moved the file using the mv command. Next I tried extracting the file using tar -zxvf apache-maven-2.2.1-bin.tar.gz. The Error Message: tar: apache-maven-2.2.1/direcoryandfile: Cannot open: No such file or directory ... apache-maven-2.2.1/lib/ext: Cannot mkdir: No such file or directory apache-maven-2.2.1/lib/ext/README.txt tar: apache-maven-2.2.1/lib/ext/README.txt: Cannot open: No such file or directory tar: Error exit delayed from previous errors From what I can tell the archive file is missing some directories or something. I tried deleting the file, redownloading the .tar.gz file from a different mirror and repeating the process. Same result. Thanks again for the help

  Read the article

• How to design a C / C++ library to be usable in many client languages?

  - by Brian Schimmel

  I'm planning to code a library that should be usable by a large number of people in on a wide spectrum of platforms. What do I have to consider to design it right? To make this questions more specific, there are four "subquestions" at the end. Choice of language Considering all the known requirements and details, I concluded that a library written in C or C++ was the way to go. I think the primary usage of my library will be in programs written in C, C++ and Java SE, but I can also think of reasons to use it from Java ME, PHP, .NET, Objective C, Python, Ruby, bash scrips, etc... Maybe I cannot target all of them, but if it's possible, I'll do it. Requirements It would be to much to describe the full purpose of my library here, but there are some aspects that might be important to this question: The library itself will start out small, but definitely will grow to enormous complexity, so it is not an option to maintain several versions in parallel. Most of the complexity will be hidden inside the library, though The library will construct an object graph that is used heavily inside. Some clients of the library will only be interested in specific attributes of specific objects, while other clients must traverse the object graph in some way Clients may change the objects, and the library must be notified thereof The library may change the objects, and the client must be notified thereof, if it already has a handle to that object The library must be multi-threaded, because it will maintain network connections to several other hosts While some requests to the library may be handled synchronously, many of them will take too long and must be processed in the background, and notify the client on success (or failure) Of course, answers are welcome no matter if they address my specific requirements, or if they answer the question in a general way that matters to a wider audience! My assumptions, so far So here are some of my assumptions and conclusions, which I gathered in the past months: Internally I can use whatever I want, e.g. C++ with operator overloading, multiple inheritance, template meta programming... as long as there is a portable compiler which handles it (think of gcc / g++) But my interface has to be a clean C interface that does not involve name mangling Also, I think my interface should only consist of functions, with basic/primitive data types (and maybe pointers) passed as parameters and return values If I use pointers, I think I should only use them to pass them back to the library, not to operate directly on the referenced memory For usage in a C++ application, I might also offer an object oriented interface (Which is also prone to name mangling, so the App must either use the same compiler, or include the library in source form) Is this also true for usage in C# ? For usage in Java SE / Java EE, the Java native interface (JNI) applies. I have some basic knowledge about it, but I should definitely double check it. Not all client languages handle multithreading well, so there should be a single thread talking to the client For usage on Java ME, there is no such thing as JNI, but I might go with Nested VM For usage in Bash scripts, there must be an executable with a command line interface For the other client languages, I have no idea For most client languages, it would be nice to have kind of an adapter interface written in that language. I think there are tools to automatically generate this for Java and some others For object oriented languages, it might be possible to create an object oriented adapter which hides the fact that the interface to the library is function based - but I don't know if its worth the effort Possible subquestions is this possible with manageable effort, or is it just too much portability? are there any good books / websites about this kind of design criteria? are any of my assumptions wrong? which open source libraries are worth studying to learn from their design / interface / souce? meta: This question is rather long, do you see any way to split it into several smaller ones? (If you reply to this, do it as a comment, not as an answer)

  Read the article

• "Could not establish secure channel for SSL/TLS" in .NET CF application on smart phone

  - by Stefan Mohr

  I have a stubborn communications issue with an application running on the .NET Compact Framework 3.5 on Windows Mobile smartphones. I am constructing a web request using this code: UTF8Encoding encoding = new System.Text.UTF8Encoding(); byte[] Data = encoding.GetBytes(HttpUtility.ConstructQueryString(parameters)); httpRequest = WebRequest.Create((domain)) as HttpWebRequest; httpRequest.Timeout = 10000000; httpRequest.ReadWriteTimeout = 10000000; httpRequest.Credentials = CredentialCache.DefaultCredentials; httpRequest.Method = "POST"; httpRequest.ContentType = "application/x-www-form-urlencoded"; httpRequest.ContentLength = Data.Length; Stream SendReq = httpRequest.GetRequestStream(); SendReq.Write(Data, 0, Data.Length); SendReq.Close(); HttpWebResponse httpResponse = (HttpWebResponse)httpRequest.GetResponse(); return httpResponse.GetResponseStream(); The web service functions by receiving a JSON-encoded document as part of the URL (eg. https://site.com/ws/sync??document={"version":"1.0.0","items":[{"item_1":"item1"}]}&user=usr&password=pw), and as a response receives another JSON document as response data. This code runs fine on all emulators and PDAs running WM 5 and 6. We have seen an issue with a couple of customers running Treo smartphones (and only on the Sprint network). We have tested the code on an identical device on the AT&T network (via DeviceAnywhere) and once again the code worked as we expected. This has to be some sort of security policy on the phone, but we've been unable to determine a workaround or diagnose it thoroughly as we cannot reproduce it in house and have had to resort to getting users to assist with running test drivers for us. When this code executes, the user's device throws the following exception: System.Net.WebException Could not establish secure channel for SSL/TLS Stack trace: at System.Net.HttpWebRequest.finishGetRequestStream() at System.Net.HttpWebRequest.GetRequestStream() at OurApp.GetResponseStream(String domain, Hashtable parameters) inner exception: System.IO.IOException Authentication failed because the remote party has closed the transport stream. Stack trace: at System.Net.SslConnectionState.ClientSideHandshake() at System.Net.SslConnectionState.PerformClientHandShake() at System.Net.Connection.connect(Object ignored) at System.Threading.ThreadPool.WorkItem.doWork(Object o) at System.Threading.Timer.ring() Examining the server Apache logs shows no hits from the user's IP - I don't think the device is even attempting to send a packet before failing. If relevant, the server is running Apache on Linux and is written using the TurboGears Python framework. The server certificate is issued by a CA and is still valid. The test driver where this error was copied from was not code signed, however the same error (without the error messages) is signed with a GeoTrust certificate so we don't believe this is a code signing issue. The application installs and launches without issue on all phones - it's just establishing this SSL connection that is breaking for these users. One significant issue in troubleshooting is that there is a substantial inconvenience each time we try out a solution (need to find a "volunteer" customer), so we're really looking for a silver bullet or a better understanding of the handshaking process so we can be reasonably confident we only need to ask the user to test it one or two more times. One final mention: we have tried the sync both over ActiveSync and also over GPRS with identical results. Any thoughts would be greatly appreciated!

  Read the article

• Modern alternatives to Java

  - by Ralph

  I have been a Java developer for 14 years and have written an enterprise-level (~500 kloc) Swing application that uses most of the standard library APIs. Recently, I have become disappointed with the progress that the language has made to "modernize" itself, and am looking for an alternative for ongoing development. I have considered moving to the .NET platform, but I have issues with using something the only runs well in Windows (I know about Mono, but that is still far behind Microsoft). I also plan on buying a new Macbook Pro as soon as Apple releases their new rumored Arrandale-based machines and want to develop in an environment that will feel "at home" in Unix/Linux. I have considered using Python or Ruby, but the standard Java library is arguably the largest of any modern language. In JVM-based languages, I looked at Groovy, but am disappointed with its performance. Rumor has it that with the soon-to-be released JDK7, with its InvokeDynamic instruction, this will improve, but I don't know how much. Groovy is also not truly a functional language, although it provides closures and some of the "functional" features on collections. It does not embrace immutability. I have narrowed my search down to two JVM-based alternatives: Scala and Clojure. Each has its strengths and weaknesses. I am looking for opinions. I am not an expert at either of these languages; I have read 2 1/2 books on Scala and am currently reading Stu Halloway's book on Clojure. Scala is strongly statically typed. I know the dynamic language folks claim that static typing is a crutch for not doing unit testing, but it does provide a mechanism for compile-time location of a whole class of errors. Scala is more concise than Java, but not as much as Clojure. Scala's inter-operation with Java seems to be better than Clojure's, in that most Java operations are easier to do in Scala than in Clojure. For example, I can find no way in Clojure to create a non-static initialization block in a class derived from a Java superclass. For example, I like the Apache commons CLI library for command line argument parsing. In Java and Scala, I can create a new Options object and add Option items to it in an initialization block as follows (Java code): final Options options = new Options() { { addOption(new Option("?", "help", false, "Show this usage information"); // other options } }; I can't figure out how to the same thing in Clojure (except by using (doit...)), although that may reflect my lack of knowledge of the language. Clojure's collections are optimized for immutability. They rarely require copy-on-write semantics. I don't know if Scala's immutable collections are implemented using similar algorithms, but Rich Hickey (Clojure's inventor) goes out of his way to explain how that language's data structures are efficient. Clojure was designed from the beginning for concurrency (as was Scala) and with modern multi-core processors, concurrency takes on more importance, but I occasionally need to write simple non-concurrent utilities, and Scala code probably runs a little faster for these applications since it discourages, but does not prohibit, "simple" mutability. One could argue that one-off utilities do not have to be super-fast, but sometimes they do tasks that take hours or days to complete. I know that there is no right answer to this "question", but I thought I would open it up for discussion. Are there other JVM-based languages that can be used for enterprise level development?

  Read the article

• (Ordered) Set Partitions in fixed-size Blocks

  - by Eugen

  Here is a function I would like to write but am unable to do so. Even if you don't / can't give a solution I would be grateful for tips. For example, I know that there is a correlation between the ordered represantions of the sum of an integer and ordered set partitions but that alone does not help me in finding the solution. So here is the description of the function I need: The Task Create an efficient* function List<int[]> createOrderedPartitions(int n_1, int n_2,..., int n_k) that returns a list of arrays of all set partions of the set {0,...,n_1+n_2+...+n_k-1} in number of arguments blocks of size (in this order) n_1,n_2,...,n_k (e.g. n_1=2, n_2=1, n_3=1 -> ({0,1},{3},{2}),...). Here is a usage example: int[] partition = createOrderedPartitions(2,1,1).get(0); partition[0]; // -> 0 partition[1]; // -> 1 partition[2]; // -> 3 partition[3]; // -> 2 Note that the number of elements in the list is (n_1+n_2+...+n_n choose n_1) * (n_2+n_3+...+n_n choose n_2) * ... * (n_k choose n_k). Also, createOrderedPartitions(1,1,1) would create the permutations of {0,1,2} and thus there would be 3! = 6 elements in the list. * by efficient I mean that you should not initially create a bigger list like all partitions and then filter out results. You should do it directly. Extra Requirements If an argument is 0 treat it as if it was not there, e.g. createOrderedPartitions(2,0,1,1) should yield the same result as createOrderedPartitions(2,1,1). But at least one argument must not be 0. Of course all arguments must be = 0. Remarks The provided pseudo code is quasi Java but the language of the solution doesn't matter. In fact, as long as the solution is fairly general and can be reproduced in other languages it is ideal. Actually, even better would be a return type of List<Tuple<Set>> (e.g. when creating such a function in Python). However, then the arguments wich have a value of 0 must not be ignored. createOrderedPartitions(2,0,2) would then create [({0,1},{},{2,3}),({0,2},{},{1,3}),({0,3},{},{1,2}),({1,2},{},{0,3}),...] Background I need this function to make my mastermind-variation bot more efficient and most of all the code more "beautiful". Take a look at the filterCandidates function in my source code. There are unnecessary / duplicate queries because I'm simply using permutations instead of specifically ordered partitions. Also, I'm just interested in how to write this function. My ideas for (ugly) "solutions" Create the powerset of {0,...,n_1+...+n_k}, filter out the subsets of size n_1, n_2 etc. and create the cartesian product of the n subsets. However this won't actually work because there would be duplicates, e.g. ({1,2},{1})... First choose n_1 of x = {0,...,n_1+n_2+...+n_n-1} and put them in the first set. Then choose n_2 of x without the n_1 chosen elements beforehand and so on. You then get for example ({0,2},{},{1,3},{4}). Of course, every possible combination must be created so ({0,4},{},{1,3},{2}), too, and so on. Seems rather hard to implement but might be possible. Research I guess this goes in the direction I want however I don't see how I can utilize it for my specific scenario. http://rosettacode.org/wiki/Combinations

  Read the article

• Are all <canvas> tag dimensions in pixels?

  - by Simon Omega

  Are all tag dimensions in pixels? I am asking because I understood them to be. But my math is broken or I am just not grasping something here. I have been doing python mostly and just jumped back into Java Scripting. If I am just doing something stupid let me know. For a game I am writing, I wanted to have a blocky gradient. I have the following: HTML <canvas id="heir"></canvas> CSS @media screen { body { font-size: 12pt } /* Game Rendering Space */ canvas { width: 640px; height: 480px; border-style: solid; border-width: 1px; } } JavaScript (Shortened) function testDraw ( thecontext ) { var myblue = 255; thecontext.save(); // Save All Settings (Before this Function was called) for (var i = 0; i < 480; i = i + 10 ) { if (myblue.toString(16).length == 1) { thecontext.fillStyle = "#00000" + myblue.toString(16); } else { thecontext.fillStyle = "#0000" + myblue.toString(16); } thecontext.fillRect(0, i, 640, 10); myblue = myblue - 2; }; thecontext.restore(); // Restore Settings to Save Point (Removing Styles, etc...) } function main () { var targetcontext = document.getElementById(“main”).getContext("2d"); testDraw(targetcontext); } To me this should produce a series of 640w by 10h pixel bars. In Google Chrome and Fire Fox I get 15 bars. To me that means ( 480 / 15 ) is 32 pixel high bars. So I change the code to: function testDraw ( thecontext ) { var myblue = 255; thecontext.save(); // Save All Settings (Before this Function was called) for (var i = 0; i < 16; i++ ) { if (myblue.toString(16).length == 1) { thecontext.fillStyle = "#00000" + myblue.toString(16); } else { thecontext.fillStyle = "#0000" + myblue.toString(16); } thecontext.fillRect(0, (i * 10), 640, 10); myblue = myblue - 10; }; thecontext.restore(); // Restore Settings to Save Point (Removing Styles, etc...) } And get a true 32 pixel height result for comparison. Other than the fact that the first code snippet has shades of blue rendering in non-visible portions of the they are measuring 32 pixels. Now back to the Original Java Code... If I inspect the tag in Chrome it reports 640 x 480. If I inspect it in Fire Fox it reports 640 x 480. BUT! Fire Fox exports the original code to png at 300 x 150 (which is 15 rows of 10). Is it some how being resized to 640 x 480 by the CSS instead of being set to a true 640 x 480? Why, how, what? O_o I confused...

  Read the article

• Two functions working but not when I put them together

  - by Error_404

  I'm really new to C(I have been learning Cuda and wanted to learn C so I can run everything together instead of generating data in Java/Python and copy/pasting it manually to my Cuda program to play with). I am trying to open a large file(20+gb) and parse some data but because I was having problems I decided to try to break down my problem first to verify I can open and read line by line a file and then in another file take a sample of the output of a line and try to parse it, then if all goes well I can simply bring them together. I managed(after a bit of a struggle) to get each part working but when I combine them together it doesn't work(I tried in both eclipse & QT creator. QT creator says it exited unexpectedly and eclipse just stops..). Here's the parsing part(it works exactly as I want): #include <string.h> #include <stdio.h> int main() { char line[1024] = "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 2, 4],"; size_t len = strlen(line); memmove(line, line+1, len-3); line[len-3] = 0; printf("%s \n",line); char str[100], *s = str, *t = NULL; strcpy(str, line); while ((t = strtok(s, " ,")) != NULL) { s = NULL; printf(":%s:\n", t); } return 0; } The data in variable line is exactly as I get if I print each line of a file. But when I copy/paste it into my main program then it doesn't work(QT shows nothing and eclipse just shows the first line). This code is just a few file IO commands that are wrapped around the commands above(everything within the while loop is the exact same as above) #include <stdio.h> #include <stdlib.h> int main() { char line[1024]; FILE *fp = fopen("/Users/me/Desktop/output.txt","r"); printf("Starting.. \n"); int count = 0; int list[30]; //items will be stored here while(fgets(line, 1024, fp) != NULL){ count++; //char line[1024] = "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 2, 4],"; size_t len = strlen(line); memmove(line, line+1, len-3); line[len-3] = 0; printf("%s \n",line); char str[100], *s = str, *t = NULL; strcpy(str, line); while ((t = strtok(s, " ,")) != NULL) { s = NULL; printf(":%s:\n", t); } } printf(" num of lines is %i \n", count); return 0; } eclipse output: Starting.. 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 2, 4 I can't figure out what I'm doing wrong. I'm using a mac with gcc 4.2, if that helps. The only thing I can think of is maybe I'm doing something wrong the way I parse the list variable so its not accessible later on(I'm not sure its a wild guess). Is there anything I'm missing?

  Read the article

• What language/framework (technology) to use for website (flash games portal)

  - by cripox

  Hello, I know there are a lot of similar questions on the net, but because I am a newbie in web development I didn't find the solution for my specific problem. I am planing on creating a flash games portal from scratch. It is a big chance that there will be big traffic from the beginning (millions of pageviews). I want to reduce the server costs as much as possible but in the same time to not be tide to an expensive contract as there is a chance that the project will not be as successfully as I want and in that case the money would be very little. The question is : what technology to use? I don't know any web dev technology yet so it doesn't matter what I will learn. My web dev experience is a little php 8 years ago, and from then I programmed in C++ / Java- game and mobile development. I like Java and C syntax and language very much and I tend to dislike dynamic typing or non robust scripting (like php)- but I can get along if these are the best choices. The candidates are now: - Grails (my best for now) Ruby on Rails Cake PHP Other technologies (Google App Engine, Python/Django etc...) I was considering at first using pure C and compiling the web app in the server- just to squeeze more from the servers, but soon I understand that this is overkill. Next my eyes came on Ruby - as there is a lot of buzz for it's easiness of use. Next I discovered Grails and looked at Java because it is said that it is "faster". But I don't know what this "Faster" really means on my needs, so here comes the first question: 1) What will be my biggest consumption on the server, other than bandwidth, for a lot of flash content requests? Is it memory? I heard that Java needs a lot of memory, but is faster. Is it CPU? I am planning to take some daily VPS.NET nodes at first, to see if there is a demand, and if the "spike" is permanent to move to a dedicated server (serverloft.com has some good offers), else to remain with less nodes. I was also considering developing in Google App Engine- cheap or free hosting to use at first - so I can test my assumption- and also very easy to use (no need for sys administration) but the costs became high if used more ( 3 million games played / month .. x mb/ each). And the issue with Google is that it looks me in this technology. My other concern is scalability (not only for traffic/users, but as adding functionality) My plans are to release a functional site in just 4 weeks (just the basics frontend and some quick basic backend - so I can be able to modify some things and add games manually) - but then to raise it and add more things to it. I am planning to take a little different approach than other portals so I need to write it from scratch (a script will not do). 2) Will Grails take much more resources than RoR or Php server wise? I heard that making it on Java stack will be hardware expensive and is overkill if you don't make a bank application. My application will not be very complex (I hope and i will try to) but will have a lot of traffic. I also took in account using CDN for files, but the cheapest CDN found was 5c/GB (vps.net) and the cost per gb on serverloft (http://www.serverloft.com/dedizierte-server/server-details.php?products=4) is only 1.79 cents/GB and comes with the other resources either. I am new to this domain (web). I am learning the ropes and searching on the web for ~half of year but don't have any really practical experience, so I know that I must have some naive thinking and other issues that i don't know from now, so please give me any advice you want regarding anything, not just the specific questions asked. And thank you so much for such great community!

  Read the article

• Creating a dynamic, extensible C# Expando Object

  - by Rick Strahl

  I love dynamic functionality in a strongly typed language because it offers us the best of both worlds. In C# (or any of the main .NET languages) we now have the dynamic type that provides a host of dynamic features for the static C# language. One place where I've found dynamic to be incredibly useful is in building extensible types or types that expose traditionally non-object data (like dictionaries) in easier to use and more readable syntax. I wrote about a couple of these for accessing old school ADO.NET DataRows and DataReaders more easily for example. These classes are dynamic wrappers that provide easier syntax and auto-type conversions which greatly simplifies code clutter and increases clarity in existing code. ExpandoObject in .NET 4.0 Another great use case for dynamic objects is the ability to create extensible objects - objects that start out with a set of static members and then can add additional properties and even methods dynamically. The .NET 4.0 framework actually includes an ExpandoObject class which provides a very dynamic object that allows you to add properties and methods on the fly and then access them again. For example with ExpandoObject you can do stuff like this:dynamic expand = new ExpandoObject(); expand.Name = "Rick"; expand.HelloWorld = (Func<string, string>) ((string name) => { return "Hello " + name; }); Console.WriteLine(expand.Name); Console.WriteLine(expand.HelloWorld("Dufus")); Internally ExpandoObject uses a Dictionary like structure and interface to store properties and methods and then allows you to add and access properties and methods easily. As cool as ExpandoObject is it has a few shortcomings too: It's a sealed type so you can't use it as a base class It only works off 'properties' in the internal Dictionary - you can't expose existing type data It doesn't serialize to XML or with DataContractSerializer/DataContractJsonSerializer Expando - A truly extensible Object ExpandoObject is nice if you just need a dynamic container for a dictionary like structure. However, if you want to build an extensible object that starts out with a set of strongly typed properties and then allows you to extend it, ExpandoObject does not work because it's a sealed class that can't be inherited. I started thinking about this very scenario for one of my applications I'm building for a customer. In this system we are connecting to various different user stores. Each user store has the same basic requirements for username, password, name etc. But then each store also has a number of extended properties that is available to each application. In the real world scenario the data is loaded from the database in a data reader and the known properties are assigned from the known fields in the database. All unknown fields are then 'added' to the expando object dynamically. In the past I've done this very thing with a separate property - Properties - just like I do for this class. But the property and dictionary syntax is not ideal and tedious to work with. I started thinking about how to represent these extra property structures. One way certainly would be to add a Dictionary, or an ExpandoObject to hold all those extra properties. But wouldn't it be nice if the application could actually extend an existing object that looks something like this as you can with the Expando object:public class User : Westwind.Utilities.Dynamic.Expando { public string Email { get; set; } public string Password { get; set; } public string Name { get; set; } public bool Active { get; set; } public DateTime? ExpiresOn { get; set; } } and then simply start extending the properties of this object dynamically? Using the Expando object I describe later you can now do the following:[TestMethod] public void UserExampleTest() { var user = new User(); // Set strongly typed properties user.Email = "[email protected]"; user.Password = "nonya123"; user.Name = "Rickochet"; user.Active = true; // Now add dynamic properties dynamic duser = user; duser.Entered = DateTime.Now; duser.Accesses = 1; // you can also add dynamic props via indexer user["NickName"] = "AntiSocialX"; duser["WebSite"] = "http://www.west-wind.com/weblog"; // Access strong type through dynamic ref Assert.AreEqual(user.Name,duser.Name); // Access strong type through indexer Assert.AreEqual(user.Password,user["Password"]); // access dyanmically added value through indexer Assert.AreEqual(duser.Entered,user["Entered"]); // access index added value through dynamic Assert.AreEqual(user["NickName"],duser.NickName); // loop through all properties dynamic AND strong type properties (true) foreach (var prop in user.GetProperties(true)) { object val = prop.Value; if (val == null) val = "null"; Console.WriteLine(prop.Key + ": " + val.ToString()); } } As you can see this code somewhat blurs the line between a static and dynamic type. You start with a strongly typed object that has a fixed set of properties. You can then cast the object to dynamic (as I discussed in my last post) and add additional properties to the object. You can also use an indexer to add dynamic properties to the object. To access the strongly typed properties you can use either the strongly typed instance, the indexer or the dynamic cast of the object. Personally I think it's kinda cool to have an easy way to access strongly typed properties by string which can make some data scenarios much easier. To access the 'dynamically added' properties you can use either the indexer on the strongly typed object, or property syntax on the dynamic cast. Using the dynamic type allows all three modes to work on both strongly typed and dynamic properties. Finally you can iterate over all properties, both dynamic and strongly typed if you chose. Lots of flexibility. Note also that by default the Expando object works against the (this) instance meaning it extends the current object. You can also pass in a separate instance to the constructor in which case that object will be used to iterate over to find properties rather than this. Using this approach provides some really interesting functionality when use the dynamic type. To use this we have to add an explicit constructor to the Expando subclass:public class User : Westwind.Utilities.Dynamic.Expando { public string Email { get; set; } public string Password { get; set; } public string Name { get; set; } public bool Active { get; set; } public DateTime? ExpiresOn { get; set; } public User() : base() { } // only required if you want to mix in seperate instance public User(object instance) : base(instance) { } } to allow the instance to be passed. When you do you can now do:[TestMethod] public void ExpandoMixinTest() { // have Expando work on Addresses var user = new User( new Address() ); // cast to dynamicAccessToPropertyTest dynamic duser = user; // Set strongly typed properties duser.Email = "[email protected]"; user.Password = "nonya123"; // Set properties on address object duser.Address = "32 Kaiea"; //duser.Phone = "808-123-2131"; // set dynamic properties duser.NonExistantProperty = "This works too"; // shows default value Address.Phone value Console.WriteLine(duser.Phone); } Using the dynamic cast in this case allows you to access *three* different 'objects': The strong type properties, the dynamically added properties in the dictionary and the properties of the instance passed in! Effectively this gives you a way to simulate multiple inheritance (which is scary - so be very careful with this, but you can do it). How Expando works Behind the scenes Expando is a DynamicObject subclass as I discussed in my last post. By implementing a few of DynamicObject's methods you can basically create a type that can trap 'property missing' and 'method missing' operations. When you access a non-existant property a known method is fired that our code can intercept and provide a value for. Internally Expando uses a custom dictionary implementation to hold the dynamic properties you might add to your expandable object. Let's look at code first. The code for the Expando type is straight forward and given what it provides relatively short. Here it is.using System; using System.Collections.Generic; using System.Linq; using System.Dynamic; using System.Reflection; namespace Westwind.Utilities.Dynamic { /// <summary> /// Class that provides extensible properties and methods. This /// dynamic object stores 'extra' properties in a dictionary or /// checks the actual properties of the instance. /// /// This means you can subclass this expando and retrieve either /// native properties or properties from values in the dictionary. /// /// This type allows you three ways to access its properties: /// /// Directly: any explicitly declared properties are accessible /// Dynamic: dynamic cast allows access to dictionary and native properties/methods /// Dictionary: Any of the extended properties are accessible via IDictionary interface /// </summary> [Serializable] public class Expando : DynamicObject, IDynamicMetaObjectProvider { /// <summary> /// Instance of object passed in /// </summary> object Instance; /// <summary> /// Cached type of the instance /// </summary> Type InstanceType; PropertyInfo[] InstancePropertyInfo { get { if (_InstancePropertyInfo == null && Instance != null) _InstancePropertyInfo = Instance.GetType().GetProperties(BindingFlags.Instance | BindingFlags.Public | BindingFlags.DeclaredOnly); return _InstancePropertyInfo; } } PropertyInfo[] _InstancePropertyInfo; /// <summary> /// String Dictionary that contains the extra dynamic values /// stored on this object/instance /// </summary> /// <remarks>Using PropertyBag to support XML Serialization of the dictionary</remarks> public PropertyBag Properties = new PropertyBag(); //public Dictionary<string,object> Properties = new Dictionary<string, object>(); /// <summary> /// This constructor just works off the internal dictionary and any /// public properties of this object. /// /// Note you can subclass Expando. /// </summary> public Expando() { Initialize(this); } /// <summary> /// Allows passing in an existing instance variable to 'extend'. /// </summary> /// <remarks> /// You can pass in null here if you don't want to /// check native properties and only check the Dictionary! /// </remarks> /// <param name="instance"></param> public Expando(object instance) { Initialize(instance); } protected virtual void Initialize(object instance) { Instance = instance; if (instance != null) InstanceType = instance.GetType(); } /// <summary> /// Try to retrieve a member by name first from instance properties /// followed by the collection entries. /// </summary> /// <param name="binder"></param> /// <param name="result"></param> /// <returns></returns> public override bool TryGetMember(GetMemberBinder binder, out object result) { result = null; // first check the Properties collection for member if (Properties.Keys.Contains(binder.Name)) { result = Properties[binder.Name]; return true; } // Next check for Public properties via Reflection if (Instance != null) { try { return GetProperty(Instance, binder.Name, out result); } catch { } } // failed to retrieve a property result = null; return false; } /// <summary> /// Property setter implementation tries to retrieve value from instance /// first then into this object /// </summary> /// <param name="binder"></param> /// <param name="value"></param> /// <returns></returns> public override bool TrySetMember(SetMemberBinder binder, object value) { // first check to see if there's a native property to set if (Instance != null) { try { bool result = SetProperty(Instance, binder.Name, value); if (result) return true; } catch { } } // no match - set or add to dictionary Properties[binder.Name] = value; return true; } /// <summary> /// Dynamic invocation method. Currently allows only for Reflection based /// operation (no ability to add methods dynamically). /// </summary> /// <param name="binder"></param> /// <param name="args"></param> /// <param name="result"></param> /// <returns></returns> public override bool TryInvokeMember(InvokeMemberBinder binder, object[] args, out object result) { if (Instance != null) { try { // check instance passed in for methods to invoke if (InvokeMethod(Instance, binder.Name, args, out result)) return true; } catch { } } result = null; return false; } /// <summary> /// Reflection Helper method to retrieve a property /// </summary> /// <param name="instance"></param> /// <param name="name"></param> /// <param name="result"></param> /// <returns></returns> protected bool GetProperty(object instance, string name, out object result) { if (instance == null) instance = this; var miArray = InstanceType.GetMember(name, BindingFlags.Public | BindingFlags.GetProperty | BindingFlags.Instance); if (miArray != null && miArray.Length > 0) { var mi = miArray[0]; if (mi.MemberType == MemberTypes.Property) { result = ((PropertyInfo)mi).GetValue(instance,null); return true; } } result = null; return false; } /// <summary> /// Reflection helper method to set a property value /// </summary> /// <param name="instance"></param> /// <param name="name"></param> /// <param name="value"></param> /// <returns></returns> protected bool SetProperty(object instance, string name, object value) { if (instance == null) instance = this; var miArray = InstanceType.GetMember(name, BindingFlags.Public | BindingFlags.SetProperty | BindingFlags.Instance); if (miArray != null && miArray.Length > 0) { var mi = miArray[0]; if (mi.MemberType == MemberTypes.Property) { ((PropertyInfo)mi).SetValue(Instance, value, null); return true; } } return false; } /// <summary> /// Reflection helper method to invoke a method /// </summary> /// <param name="instance"></param> /// <param name="name"></param> /// <param name="args"></param> /// <param name="result"></param> /// <returns></returns> protected bool InvokeMethod(object instance, string name, object[] args, out object result) { if (instance == null) instance = this; // Look at the instanceType var miArray = InstanceType.GetMember(name, BindingFlags.InvokeMethod | BindingFlags.Public | BindingFlags.Instance); if (miArray != null && miArray.Length > 0) { var mi = miArray[0] as MethodInfo; result = mi.Invoke(Instance, args); return true; } result = null; return false; } /// <summary> /// Convenience method that provides a string Indexer /// to the Properties collection AND the strongly typed /// properties of the object by name. /// /// // dynamic /// exp["Address"] = "112 nowhere lane"; /// // strong /// var name = exp["StronglyTypedProperty"] as string; /// </summary> /// <remarks> /// The getter checks the Properties dictionary first /// then looks in PropertyInfo for properties. /// The setter checks the instance properties before /// checking the Properties dictionary. /// </remarks> /// <param name="key"></param> /// /// <returns></returns> public object this[string key] { get { try { // try to get from properties collection first return Properties[key]; } catch (KeyNotFoundException ex) { // try reflection on instanceType object result = null; if (GetProperty(Instance, key, out result)) return result; // nope doesn't exist throw; } } set { if (Properties.ContainsKey(key)) { Properties[key] = value; return; } // check instance for existance of type first var miArray = InstanceType.GetMember(key, BindingFlags.Public | BindingFlags.GetProperty); if (miArray != null && miArray.Length > 0) SetProperty(Instance, key, value); else Properties[key] = value; } } /// <summary> /// Returns and the properties of /// </summary> /// <param name="includeProperties"></param> /// <returns></returns> public IEnumerable<KeyValuePair<string,object>> GetProperties(bool includeInstanceProperties = false) { if (includeInstanceProperties && Instance != null) { foreach (var prop in this.InstancePropertyInfo) yield return new KeyValuePair<string, object>(prop.Name, prop.GetValue(Instance, null)); } foreach (var key in this.Properties.Keys) yield return new KeyValuePair<string, object>(key, this.Properties[key]); } /// <summary> /// Checks whether a property exists in the Property collection /// or as a property on the instance /// </summary> /// <param name="item"></param> /// <returns></returns> public bool Contains(KeyValuePair<string, object> item, bool includeInstanceProperties = false) { bool res = Properties.ContainsKey(item.Key); if (res) return true; if (includeInstanceProperties && Instance != null) { foreach (var prop in this.InstancePropertyInfo) { if (prop.Name == item.Key) return true; } } return false; } } } Although the Expando class supports an indexer, it doesn't actually implement IDictionary or even IEnumerable. It only provides the indexer and Contains() and GetProperties() methods, that work against the Properties dictionary AND the internal instance. The reason for not implementing IDictionary is that a) it doesn't add much value since you can access the Properties dictionary directly and that b) I wanted to keep the interface to class very lean so that it can serve as an entity type if desired. Implementing these IDictionary (or even IEnumerable) causes LINQ extension methods to pop up on the type which obscures the property interface and would only confuse the purpose of the type. IDictionary and IEnumerable are also problematic for XML and JSON Serialization - the XML Serializer doesn't serialize IDictionary<string,object>, nor does the DataContractSerializer. The JavaScriptSerializer does serialize, but it treats the entire object like a dictionary and doesn't serialize the strongly typed properties of the type, only the dictionary values which is also not desirable. Hence the decision to stick with only implementing the indexer to support the user["CustomProperty"] functionality and leaving iteration functions to the publicly exposed Properties dictionary. Note that the Dictionary used here is a custom PropertyBag class I created to allow for serialization to work. One important aspect for my apps is that whatever custom properties get added they have to be accessible to AJAX clients since the particular app I'm working on is a SIngle Page Web app where most of the Web access is through JSON AJAX calls. PropertyBag can serialize to XML and one way serialize to JSON using the JavaScript serializer (not the DCS serializers though). The key components that make Expando work in this code are the Properties Dictionary and the TryGetMember() and TrySetMember() methods. The Properties collection is public so if you choose you can explicitly access the collection to get better performance or to manipulate the members in internal code (like loading up dynamic values form a database). Notice that TryGetMember() and TrySetMember() both work against the dictionary AND the internal instance to retrieve and set properties. This means that user["Name"] works against native properties of the object as does user["Name"] = "RogaDugDog". What's your Use Case? This is still an early prototype but I've plugged it into one of my customer's applications and so far it's working very well. The key features for me were the ability to easily extend the type with values coming from a database and exposing those values in a nice and easy to use manner. I'm also finding that using this type of object for ViewModels works very well to add custom properties to view models. I suspect there will be lots of uses for this - I've been using the extra dictionary approach to extensibility for years - using a dynamic type to make the syntax cleaner is just a bonus here. What can you think of to use this for? Resources Source Code and Tests (GitHub) Also integrated in Westwind.Utilities of the West Wind Web Toolkit West Wind Utilities NuGet© Rick Strahl, West Wind Technologies, 2005-2012Posted in CSharp  .NET  Dynamic Types   Tweet !function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs"); (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/plusone.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();

  Read the article

• VS 2010 SP1 and SQL CE

  - by ScottGu

  Last month we released the Beta of VS 2010 Service Pack 1 (SP1).  You can learn more about the VS 2010 SP1 Beta from Jason Zander’s two blog posts about it, and from Scott Hanselman’s blog post that covers some of the new capabilities enabled with it.   You can download and install the VS 2010 SP1 Beta here. Last week I blogged about the new Visual Studio support for IIS Express that we are adding with VS 2010 SP1. In today’s post I’m going to talk about the new VS 2010 SP1 tooling support for SQL CE, and walkthrough some of the cool scenarios it enables.  SQL CE – What is it and why should you care? SQL CE is a free, embedded, database engine that enables easy database storage. No Database Installation Required SQL CE does not require you to run a setup or install a database server in order to use it.  You can simply copy the SQL CE binaries into the \bin directory of your ASP.NET application, and then your web application can use it as a database engine.  No setup or extra security permissions are required for it to run. You do not need to have an administrator account on the machine. Just copy your web application onto any server and it will work. This is true even of medium-trust applications running in a web hosting environment. SQL CE runs in-memory within your ASP.NET application and will start-up when you first access a SQL CE database, and will automatically shutdown when your application is unloaded.  SQL CE databases are stored as files that live within the \App_Data folder of your ASP.NET Applications. Works with Existing Data APIs SQL CE 4 works with existing .NET-based data APIs, and supports a SQL Server compatible query syntax.  This means you can use existing data APIs like ADO.NET, as well as use higher-level ORMs like Entity Framework and NHibernate with SQL CE.  This enables you to use the same data programming skills and data APIs you know today. Supports Development, Testing and Production Scenarios SQL CE can be used for development scenarios, testing scenarios, and light production usage scenarios.  With the SQL CE 4 release we’ve done the engineering work to ensure that SQL CE won’t crash or deadlock when used in a multi-threaded server scenario (like ASP.NET).  This is a big change from previous releases of SQL CE – which were designed for client-only scenarios and which explicitly blocked running in web-server environments.  Starting with SQL CE 4 you can use it in a web-server as well. There are no license restrictions with SQL CE.  It is also totally free. Easy Migration to SQL Server SQL CE is an embedded database – which makes it ideal for development, testing, and light-usage scenarios.  For high-volume sites and applications you’ll probably want to migrate your database to use SQL Server Express (which is free), SQL Server or SQL Azure.  These servers enable much better scalability, more development features (including features like Stored Procedures – which aren’t supported with SQL CE), as well as more advanced data management capabilities. We’ll ship migration tools that enable you to optionally take SQL CE databases and easily upgrade them to use SQL Server Express, SQL Server, or SQL Azure.  You will not need to change your code when upgrading a SQL CE database to SQL Server or SQL Azure.  Our goal is to enable you to be able to simply change the database connection string in your web.config file and have your application just work. New Tooling Support for SQL CE in VS 2010 SP1 VS 2010 SP1 includes much improved tooling support for SQL CE, and adds support for using SQL CE within ASP.NET projects for the first time.  With VS 2010 SP1 you can now: Create new SQL CE Databases Edit and Modify SQL CE Database Schema and Indexes Populate SQL CE Databases within Data Use the Entity Framework (EF) designer to create model layers against SQL CE databases Use EF Code First to define model layers in code, then create a SQL CE database from them, and optionally edit the DB with VS Deploy SQL CE databases to remote servers using Web Deploy and optionally convert them to full SQL Server databases You can take advantage of all of the above features from within both ASP.NET Web Forms and ASP.NET MVC based projects. Download You can enable SQL CE tooling support within VS 2010 by first installing VS 2010 SP1 (beta). Once SP1 is installed, you’ll also then need to install the SQL CE Tools for Visual Studio download.  This is a separate download that enables the SQL CE tooling support for VS 2010 SP1. Walkthrough of Two Scenarios In this blog post I’m going to walkthrough how you can take advantage of SQL CE and VS 2010 SP1 using both an ASP.NET Web Forms and an ASP.NET MVC based application. Specifically, we’ll walkthrough: How to create a SQL CE database using VS 2010 SP1, then use the EF4 visual designers in Visual Studio to construct a model layer from it, and then display and edit the data using an ASP.NET GridView control. How to use an EF Code First approach to define a model layer using POCO classes and then have EF Code-First “auto-create” a SQL CE database for us based on our model classes.  We’ll then look at how we can use the new VS 2010 SP1 support for SQL CE to inspect the database that was created, populate it with data, and later make schema changes to it.  We’ll do all this within the context of an ASP.NET MVC based application. You can follow the two walkthroughs below on your own machine by installing VS 2010 SP1 (beta) and then installing the SQL CE Tools for Visual Studio download (which is a separate download that enables SQL CE tooling support for VS 2010 SP1). Walkthrough 1: Create a SQL CE Database, Create EF Model Classes, Edit the Data with a GridView This first walkthrough will demonstrate how to create and define a SQL CE database within an ASP.NET Web Form application.  We’ll then build an EF model layer for it and use that model layer to enable data editing scenarios with an <asp:GridView> control. Step 1: Create a new ASP.NET Web Forms Project We’ll begin by using the File->New Project menu command within Visual Studio to create a new ASP.NET Web Forms project.  We’ll use the “ASP.NET Web Application” project template option so that it has a default UI skin implemented: Step 2: Create a SQL CE Database Right click on the “App_Data” folder within the created project and choose the “Add->New Item” menu command: This will bring up the “Add Item” dialog box.  Select the “SQL Server Compact 4.0 Local Database” item (new in VS 2010 SP1) and name the database file to create “Store.sdf”: Note that SQL CE database files have a .sdf filename extension. Place them within the /App_Data folder of your ASP.NET application to enable easy deployment. When we clicked the “Add” button above a Store.sdf file was added to our project: Step 3: Adding a “Products” Table Double-clicking the “Store.sdf” database file will open it up within the Server Explorer tab.  Since it is a new database there are no tables within it: Right click on the “Tables” icon and choose the “Create Table” menu command to create a new database table.  We’ll name the new table “Products” and add 4 columns to it.  We’ll mark the first column as a primary key (and make it an identify column so that its value will automatically increment with each new row): When we click “ok” our new Products table will be created in the SQL CE database. Step 4: Populate with Data Once our Products table is created it will show up within the Server Explorer.  We can right-click it and choose the “Show Table Data” menu command to edit its data: Let’s add a few sample rows of data to it: Step 5: Create an EF Model Layer We have a SQL CE database with some data in it – let’s now create an EF Model Layer that will provide a way for us to easily query and update data within it. Let’s right-click on our project and choose the “Add->New Item” menu command.  This will bring up the “Add New Item” dialog – select the “ADO.NET Entity Data Model” item within it and name it “Store.edmx” This will add a new Store.edmx item to our solution explorer and launch a wizard that allows us to quickly create an EF model: Select the “Generate From Database” option above and click next.  Choose to use the Store.sdf SQL CE database we just created and then click next again.  The wizard will then ask you what database objects you want to import into your model.  Let’s choose to import the “Products” table we created earlier: When we click the “Finish” button Visual Studio will open up the EF designer.  It will have a Product entity already on it that maps to the “Products” table within our SQL CE database: The VS 2010 SP1 EF designer works exactly the same with SQL CE as it does already with SQL Server and SQL Express.  The Product entity above will be persisted as a class (called “Product”) that we can programmatically work against within our ASP.NET application. Step 6: Compile the Project Before using your model layer you’ll need to build your project.  Do a Ctrl+Shift+B to compile the project, or use the Build->Build Solution menu command. Step 7: Create a Page that Uses our EF Model Layer Let’s now create a simple ASP.NET Web Form that contains a GridView control that we can use to display and edit the our Products data (via the EF Model Layer we just created). Right-click on the project and choose the Add->New Item command.  Select the “Web Form from Master Page” item template, and name the page you create “Products.aspx”.  Base the master page on the “Site.Master” template that is in the root of the project. Add an <h2>Products</h2> heading the new Page, and add an <asp:gridview> control within it: Then click the “Design” tab to switch into design-view. Select the GridView control, and then click the top-right corner to display the GridView’s “Smart Tasks” UI: Choose the “New data source…” drop down option above.  This will bring up the below dialog which allows you to pick your Data Source type: Select the “Entity” data source option – which will allow us to easily connect our GridView to the EF model layer we created earlier.  This will bring up another dialog that allows us to pick our model layer: Select the “StoreEntities” option in the dropdown – which is the EF model layer we created earlier.  Then click next – which will allow us to pick which entity within it we want to bind to: Select the “Products” entity in the above dialog – which indicates that we want to bind against the “Product” entity class we defined earlier.  Then click the “Enable automatic updates” checkbox to ensure that we can both query and update Products.  When you click “Finish” VS will wire-up an <asp:EntityDataSource> to your <asp:GridView> control: The last two steps we’ll do will be to click the “Enable Editing” checkbox on the Grid (which will cause the Grid to display an “Edit” link on each row) and (optionally) use the Auto Format dialog to pick a UI template for the Grid. Step 8: Run the Application Let’s now run our application and browse to the /Products.aspx page that contains our GridView.  When we do so we’ll see a Grid UI of the Products within our SQL CE database. Clicking the “Edit” link for any of the rows will allow us to edit their values: When we click “Update” the GridView will post back the values, persist them through our EF Model Layer, and ultimately save them within our SQL CE database. Learn More about using EF with ASP.NET Web Forms Read this tutorial series on the http://asp.net site to learn more about how to use EF with ASP.NET Web Forms.  The tutorial series uses SQL Express as the database – but the nice thing is that all of the same steps/concepts can also now also be done with SQL CE.   Walkthrough 2: Using EF Code-First with SQL CE and ASP.NET MVC 3 We used a database-first approach with the sample above – where we first created the database, and then used the EF designer to create model classes from the database.  In addition to supporting a designer-based development workflow, EF also enables a more code-centric option which we call “code first development”.  Code-First Development enables a pretty sweet development workflow.  It enables you to: Define your model objects by simply writing “plain old classes” with no base classes or visual designer required Use a “convention over configuration” approach that enables database persistence without explicitly configuring anything Optionally override the convention-based persistence and use a fluent code API to fully customize the persistence mapping Optionally auto-create a database based on the model classes you define – allowing you to start from code first I’ve done several blog posts about EF Code First in the past – I really think it is great.  The good news is that it also works very well with SQL CE. The combination of SQL CE, EF Code First, and the new VS tooling support for SQL CE, enables a pretty nice workflow.  Below is a simple example of how you can use them to build a simple ASP.NET MVC 3 application. Step 1: Create a new ASP.NET MVC 3 Project We’ll begin by using the File->New Project menu command within Visual Studio to create a new ASP.NET MVC 3 project.  We’ll use the “Internet Project” template so that it has a default UI skin implemented: Step 2: Use NuGet to Install EFCodeFirst Next we’ll use the NuGet package manager (automatically installed by ASP.NET MVC 3) to add the EFCodeFirst library to our project.  We’ll use the Package Manager command shell to do this.  Bring up the package manager console within Visual Studio by selecting the View->Other Windows->Package Manager Console menu command.  Then type: install-package EFCodeFirst within the package manager console to download the EFCodeFirst library and have it be added to our project: When we enter the above command, the EFCodeFirst library will be downloaded and added to our application: Step 3: Build Some Model Classes Using a “code first” based development workflow, we will create our model classes first (even before we have a database).  We create these model classes by writing code. For this sample, we will right click on the “Models” folder of our project and add the below three classes to our project: The “Dinner” and “RSVP” model classes above are “plain old CLR objects” (aka POCO).  They do not need to derive from any base classes or implement any interfaces, and the properties they expose are standard .NET data-types.  No data persistence attributes or data code has been added to them.   The “NerdDinners” class derives from the DbContext class (which is supplied by EFCodeFirst) and handles the retrieval/persistence of our Dinner and RSVP instances from a database. Step 4: Listing Dinners We’ve written all of the code necessary to implement our model layer for this simple project.  Let’s now expose and implement the URL: /Dinners/Upcoming within our project.  We’ll use it to list upcoming dinners that happen in the future. We’ll do this by right-clicking on our “Controllers” folder and select the “Add->Controller” menu command.  We’ll name the Controller we want to create “DinnersController”.  We’ll then implement an “Upcoming” action method within it that lists upcoming dinners using our model layer above.  We will use a LINQ query to retrieve the data and pass it to a View to render with the code below: We’ll then right-click within our Upcoming method and choose the “Add-View” menu command to create an “Upcoming” view template that displays our dinners.  We’ll use the “empty” template option within the “Add View” dialog and write the below view template using Razor: Step 4: Configure our Project to use a SQL CE Database We have finished writing all of our code – our last step will be to configure a database connection-string to use. We will point our NerdDinners model class to a SQL CE database by adding the below <connectionString> to the web.config file at the top of our project: EF Code First uses a default convention where context classes will look for a connection-string that matches the DbContext class name.  Because we created a “NerdDinners” class earlier, we’ve also named our connectionstring “NerdDinners”.  Above we are configuring our connection-string to use SQL CE as the database, and telling it that our SQL CE database file will live within the \App_Data directory of our ASP.NET project. Step 5: Running our Application Now that we’ve built our application, let’s run it! We’ll browse to the /Dinners/Upcoming URL – doing so will display an empty list of upcoming dinners: You might ask – but where did it query to get the dinners from? We didn’t explicitly create a database?!? One of the cool features that EF Code-First supports is the ability to automatically create a database (based on the schema of our model classes) when the database we point it at doesn’t exist.  Above we configured  EF Code-First to point at a SQL CE database in the \App_Data\ directory of our project.  When we ran our application, EF Code-First saw that the SQL CE database didn’t exist and automatically created it for us. Step 6: Using VS 2010 SP1 to Explore our newly created SQL CE Database Click the “Show all Files” icon within the Solution Explorer and you’ll see the “NerdDinners.sdf” SQL CE database file that was automatically created for us by EF code-first within the \App_Data\ folder: We can optionally right-click on the file and “Include in Project" to add it to our solution: We can also double-click the file (regardless of whether it is added to the project) and VS 2010 SP1 will open it as a database we can edit within the “Server Explorer” tab of the IDE. Below is the view we get when we double-click our NerdDinners.sdf SQL CE file.  We can drill in to see the schema of the Dinners and RSVPs tables in the tree explorer.  Notice how two tables - Dinners and RSVPs – were automatically created for us within our SQL CE database.  This was done by EF Code First when we accessed the NerdDinners class by running our application above: We can right-click on a Table and use the “Show Table Data” command to enter some upcoming dinners in our database: We’ll use the built-in editor that VS 2010 SP1 supports to populate our table data below: And now when we hit “refresh” on the /Dinners/Upcoming URL within our browser we’ll see some upcoming dinners show up: Step 7: Changing our Model and Database Schema Let’s now modify the schema of our model layer and database, and walkthrough one way that the new VS 2010 SP1 Tooling support for SQL CE can make this easier.  With EF Code-First you typically start making database changes by modifying the model classes.  For example, let’s add an additional string property called “UrlLink” to our “Dinner” class.  We’ll use this to point to a link for more information about the event: Now when we re-run our project, and visit the /Dinners/Upcoming URL we’ll see an error thrown: We are seeing this error because EF Code-First automatically created our database, and by default when it does this it adds a table that helps tracks whether the schema of our database is in sync with our model classes.  EF Code-First helpfully throws an error when they become out of sync – making it easier to track down issues at development time that you might otherwise only find (via obscure errors) at runtime.  Note that if you do not want this feature you can turn it off by changing the default conventions of your DbContext class (in this case our NerdDinners class) to not track the schema version. Our model classes and database schema are out of sync in the above example – so how do we fix this?  There are two approaches you can use today: Delete the database and have EF Code First automatically re-create the database based on the new model class schema (losing the data within the existing DB) Modify the schema of the existing database to make it in sync with the model classes (keeping/migrating the data within the existing DB) There are a couple of ways you can do the second approach above.  Below I’m going to show how you can take advantage of the new VS 2010 SP1 Tooling support for SQL CE to use a database schema tool to modify our database structure.  We are also going to be supporting a “migrations” feature with EF in the future that will allow you to automate/script database schema migrations programmatically. Step 8: Modify our SQL CE Database Schema using VS 2010 SP1 The new SQL CE Tooling support within VS 2010 SP1 makes it easy to modify the schema of our existing SQL CE database.  To do this we’ll right-click on our “Dinners” table and choose the “Edit Table Schema” command: This will bring up the below “Edit Table” dialog.  We can rename, change or delete any of the existing columns in our table, or click at the bottom of the column listing and type to add a new column.  Below I’ve added a new “UrlLink” column of type “nvarchar” (since our property is a string): When we click ok our database will be updated to have the new column and our schema will now match our model classes. Because we are manually modifying our database schema, there is one additional step we need to take to let EF Code-First know that the database schema is in sync with our model classes.  As i mentioned earlier, when a database is automatically created by EF Code-First it adds a “EdmMetadata” table to the database to track schema versions (and hash our model classes against them to detect mismatches between our model classes and the database schema): Since we are manually updating and maintaining our database schema, we don’t need this table – and can just delete it: This will leave us with just the two tables that correspond to our model classes: And now when we re-run our /Dinners/Upcoming URL it will display the dinners correctly: One last touch we could do would be to update our view to check for the new UrlLink property and render a <a> link to it if an event has one: And now when we refresh our /Dinners/Upcoming we will see hyperlinks for the events that have a UrlLink stored in the database: Summary SQL CE provides a free, embedded, database engine that you can use to easily enable database storage.  With SQL CE 4 you can now take advantage of it within ASP.NET projects and applications (both Web Forms and MVC). VS 2010 SP1 provides tooling support that enables you to easily create, edit and modify SQL CE databases – as well as use the standard EF designer against them.  This allows you to re-use your existing skills and data knowledge while taking advantage of an embedded database option.  This is useful both for small applications (where you don’t need the scalability of a full SQL Server), as well as for development and testing scenarios – where you want to be able to rapidly develop/test your application without having a full database instance.  SQL CE makes it easy to later migrate your data to a full SQL Server or SQL Azure instance if you want to – without having to change any code in your application.  All we would need to change in the above two scenarios is the <connectionString> value within the web.config file in order to have our code run against a full SQL Server.  This provides the flexibility to scale up your application starting from a small embedded database solution as needed. Hope this helps, Scott P.S. In addition to blogging, I am also now using Twitter for quick updates and to share links. Follow me at: twitter.com/scottgu

  Read the article

• Top things web developers should know about the Visual Studio 2013 release

  - by Jon Galloway

  ASP.NET and Web Tools for Visual Studio 2013 Release NotesASP.NET and Web Tools for Visual Studio 2013 Release NotesSummary for lazy readers: Visual Studio 2013 is now available for download on the Visual Studio site and on MSDN subscriber downloads) Visual Studio 2013 installs side by side with Visual Studio 2012 and supports round-tripping between Visual Studio versions, so you can try it out without committing to a switch Visual Studio 2013 ships with the new version of ASP.NET, which includes ASP.NET MVC 5, ASP.NET Web API 2, Razor 3, Entity Framework 6 and SignalR 2.0 The new releases ASP.NET focuses on One ASP.NET, so core features and web tools work the same across the platform (e.g. adding ASP.NET MVC controllers to a Web Forms application) New core features include new templates based on Bootstrap, a new scaffolding system, and a new identity system Visual Studio 2013 is an incredible editor for web files, including HTML, CSS, JavaScript, Markdown, LESS, Coffeescript, Handlebars, Angular, Ember, Knockdown, etc. Top links: Visual Studio 2013 content on the ASP.NET site are in the standard new releases area: http://www.asp.net/vnext ASP.NET and Web Tools for Visual Studio 2013 Release Notes Short intro videos on the new Visual Studio web editor features from Scott Hanselman and Mads Kristensen Announcing release of ASP.NET and Web Tools for Visual Studio 2013 post on the official .NET Web Development and Tools Blog Scott Guthrie's post: Announcing the Release of Visual Studio 2013 and Great Improvements to ASP.NET and Entity Framework Okay, for those of you who are still with me, let's dig in a bit. Quick web dev notes on downloading and installing Visual Studio 2013 I found Visual Studio 2013 to be a pretty fast install. According to Brian Harry's release post, installing over pre-release versions of Visual Studio is supported.  I've installed the release version over pre-release versions, and it worked fine. If you're only going to be doing web development, you can speed up the install if you just select Web Developer tools. Of course, as a good Microsoft employee, I'll mention that you might also want to install some of those other features, like the Store apps for Windows 8 and the Windows Phone 8.0 SDK, but they do download and install a lot of other stuff (e.g. the Windows Phone SDK sets up Hyper-V and downloads several GB's of VM's). So if you're planning just to do web development for now, you can pick just the Web Developer Tools and install the other stuff later. If you've got a fast internet connection, I recommend using the web installer instead of downloading the ISO. The ISO includes all the features, whereas the web installer just downloads what you're installing. Visual Studio 2013 development settings and color theme When you start up Visual Studio, it'll prompt you to pick some defaults. These are totally up to you -whatever suits your development style - and you can change them later. As I said, these are completely up to you. I recommend either the Web Development or Web Development (Code Only) settings. The only real difference is that Code Only hides the toolbars, and you can switch between them using Tools / Import and Export Settings / Reset. Web Development settings Web Development (code only) settings Usually I've just gone with Web Development (code only) in the past because I just want to focus on the code, although the Standard toolbar does make it easier to switch default web browsers. More on that later. Color theme Sigh. Okay, everyone's got their favorite colors. I alternate between Light and Dark depending on my mood, and I personally like how the low contrast on the window chrome in those themes puts the emphasis on my code rather than the tabs and toolbars. I know some people got pretty worked up over that, though, and wanted the blue theme back. I personally don't like it - it reminds me of ancient versions of Visual Studio that I don't want to think about anymore. So here's the thing: if you install Visual Studio Ultimate, it defaults to Blue. The other versions default to Light. If you use Blue, I won't criticize you - out loud, that is. You can change themes really easily - either Tools / Options / Environment / General, or the smart way: ctrl+q for quick launch, then type Theme and hit enter. Signing in During the first run, you'll be prompted to sign in. You don't have to - you can click the "Not now, maybe later" link at the bottom of that dialog. I recommend signing in, though. It's not hooked in with licensing or tracking the kind of code you write to sell you components. It is doing good things, like  syncing your Visual Studio settings between computers. More about that here. So, you don't have to, but I sure do. Overview of shiny new things in ASP.NET land There are a lot of good new things in ASP.NET. I'll list some of my favorite here, but you can read more on the ASP.NET site. One ASP.NET You've heard us talk about this for a while. The idea is that options are good, but choice can be a burden. When you start a new ASP.NET project, why should you have to make a tough decision - with long-term consequences - about how your application will work? If you want to use ASP.NET Web Forms, but have the option of adding in ASP.NET MVC later, why should that be hard? It's all ASP.NET, right? Ideally, you'd just decide that you want to use ASP.NET to build sites and services, and you could use the appropriate tools (the green blocks below) as you needed them. So, here it is. When you create a new ASP.NET application, you just create an ASP.NET application. Next, you can pick from some templates to get you started... but these are different. They're not "painful decision" templates, they're just some starting pieces. And, most importantly, you can mix and match. I can pick a "mostly" Web Forms template, but include MVC and Web API folders and core references. If you've tried to mix and match in the past, you're probably aware that it was possible, but not pleasant. ASP.NET MVC project files contained special project type GUIDs, so you'd only get controller scaffolding support in a Web Forms project if you manually edited the csproj file. Features in one stack didn't work in others. Project templates were painful choices. That's no longer the case. Hooray! I just did a demo in a presentation last week where I created a new Web Forms + MVC + Web API site, built a model, scaffolded MVC and Web API controllers with EF Code First, add data in the MVC view, viewed it in Web API, then added a GridView to the Web Forms Default.aspx page and bound it to the Model. In about 5 minutes. Sure, it's a simple example, but it's great to be able to share code and features across the whole ASP.NET family. Authentication In the past, authentication was built into the templates. So, for instance, there was an ASP.NET MVC 4 Intranet Project template which created a new ASP.NET MVC 4 application that was preconfigured for Windows Authentication. All of that authentication stuff was built into each template, so they varied between the stacks, and you couldn't reuse them. You didn't see a lot of changes to the authentication options, since they required big changes to a bunch of project templates. Now, the new project dialog includes a common authentication experience. When you hit the Change Authentication button, you get some common options that work the same way regardless of the template or reference settings you've made. These options work on all ASP.NET frameworks, and all hosting environments (IIS, IIS Express, or OWIN for self-host) The default is Individual User Accounts: This is the standard "create a local account, using username / password or OAuth" thing; however, it's all built on the new Identity system. More on that in a second. The one setting that has some configuration to it is Organizational Accounts, which lets you configure authentication using Active Directory, Windows Azure Active Directory, or Office 365. Identity There's a new identity system. We've taken the best parts of the previous ASP.NET Membership and Simple Identity systems, rolled in a lot of feedback and made big enhancements to support important developer concerns like unit testing and extensiblity. I've written long posts about ASP.NET identity, and I'll do it again. Soon. This is not that post. The short version is that I think we've finally got just the right Identity system. Some of my favorite features: There are simple, sensible defaults that work well - you can File / New / Run / Register / Login, and everything works. It supports standard username / password as well as external authentication (OAuth, etc.). It's easy to customize without having to re-implement an entire provider. It's built using pluggable pieces, rather than one large monolithic system. It's built using interfaces like IUser and IRole that allow for unit testing, dependency injection, etc. You can easily add user profile data (e.g. URL, twitter handle, birthday). You just add properties to your ApplicationUser model and they'll automatically be persisted. Complete control over how the identity data is persisted. By default, everything works with Entity Framework Code First, but it's built to support changes from small (modify the schema) to big (use another ORM, store your data in a document database or in the cloud or in XML or in the EXIF data of your desktop background or whatever). It's configured via OWIN. More on OWIN and Katana later, but the fact that it's built using OWIN means it's portable. You can find out more in the Authentication and Identity section of the ASP.NET site (and lots more content will be going up there soon). New Bootstrap based project templates The new project templates are built using Bootstrap 3. Bootstrap (formerly Twitter Bootstrap) is a front-end framework that brings a lot of nice benefits: It's responsive, so your projects will automatically scale to device width using CSS media queries. For example, menus are full size on a desktop browser, but on narrower screens you automatically get a mobile-friendly menu. The built-in Bootstrap styles make your standard page elements (headers, footers, buttons, form inputs, tables etc.) look nice and modern. Bootstrap is themeable, so you can reskin your whole site by dropping in a new Bootstrap theme. Since Bootstrap is pretty popular across the web development community, this gives you a large and rapidly growing variety of templates (free and paid) to choose from. Bootstrap also includes a lot of very useful things: components (like progress bars and badges), useful glyphicons, and some jQuery plugins for tooltips, dropdowns, carousels, etc.). Here's a look at how the responsive part works. When the page is full screen, the menu and header are optimized for a wide screen display: When I shrink the page down (this is all based on page width, not useragent sniffing) the menu turns into a nice mobile-friendly dropdown: For a quick example, I grabbed a new free theme off bootswatch.com. For simple themes, you just need to download the boostrap.css file and replace the /content/bootstrap.css file in your project. Now when I refresh the page, I've got a new theme: Scaffolding The big change in scaffolding is that it's one system that works across ASP.NET. You can create a new Empty Web project or Web Forms project and you'll get the Scaffold context menus. For release, we've got MVC 5 and Web API 2 controllers. We had a preview of Web Forms scaffolding in the preview releases, but they weren't fully baked for RTM. Look for them in a future update, expected pretty soon. This scaffolding system wasn't just changed to work across the ASP.NET frameworks, it's also built to enable future extensibility. That's not in this release, but should also hopefully be out soon. Project Readme page This is a small thing, but I really like it. When you create a new project, you get a Project_Readme.html page that's added to the root of your project and opens in the Visual Studio built-in browser. I love it. A long time ago, when you created a new project we just dumped it on you and left you scratching your head about what to do next. Not ideal. Then we started adding a bunch of Getting Started information to the new project templates. That told you what to do next, but you had to delete all of that stuff out of your website. It doesn't belong there. Not ideal. This is a simple HTML file that's not integrated into your project code at all. You can delete it if you want. But, it shows a lot of helpful links that are current for the project you just created. In the future, if we add new wacky project types, they can create readme docs with specific information on how to do appropriately wacky things. Side note: I really like that they used the internal browser in Visual Studio to show this content rather than popping open an HTML page in the default browser. I hate that. It's annoying. If you're doing that, I hope you'll stop. What if some unnamed person has 40 or 90 tabs saved in their browser session? When you pop open your "Thanks for installing my Visual Studio extension!" page, all eleventy billion tabs start up and I wish I'd never installed your thing. Be like these guys and pop stuff Visual Studio specific HTML docs in the Visual Studio browser. ASP.NET MVC 5 The biggest change with ASP.NET MVC 5 is that it's no longer a separate project type. It integrates well with the rest of ASP.NET. In addition to that and the other common features we've already looked at (Bootstrap templates, Identity, authentication), here's what's new for ASP.NET MVC. Attribute routing ASP.NET MVC now supports attribute routing, thanks to a contribution by Tim McCall, the author of http://attributerouting.net. With attribute routing you can specify your routes by annotating your actions and controllers. This supports some pretty complex, customized routing scenarios, and it allows you to keep your route information right with your controller actions if you'd like. Here's a controller that includes an action whose method name is Hiding, but I've used AttributeRouting to configure it to /spaghetti/with-nesting/where-is-waldo public class SampleController : Controller { [Route("spaghetti/with-nesting/where-is-waldo")] public string Hiding() { return "You found me!"; } } I enable that in my RouteConfig.cs, and I can use that in conjunction with my other MVC routes like this: public class RouteConfig { public static void RegisterRoutes(RouteCollection routes) { routes.IgnoreRoute("{resource}.axd/{*pathInfo}"); routes.MapMvcAttributeRoutes(); routes.MapRoute( name: "Default", url: "{controller}/{action}/{id}", defaults: new { controller = "Home", action = "Index", id = UrlParameter.Optional } ); } } You can read more about Attribute Routing in ASP.NET MVC 5 here. Filter enhancements There are two new additions to filters: Authentication Filters and Filter Overrides. Authentication filters are a new kind of filter in ASP.NET MVC that run prior to authorization filters in the ASP.NET MVC pipeline and allow you to specify authentication logic per-action, per-controller, or globally for all controllers. Authentication filters process credentials in the request and provide a corresponding principal. Authentication filters can also add authentication challenges in response to unauthorized requests. Override filters let you change which filters apply to a given action method or controller. Override filters specify a set of filter types that should not be run for a given scope (action or controller). This allows you to configure filters that apply globally but then exclude certain global filters from applying to specific actions or controllers. ASP.NET Web API 2 ASP.NET Web API 2 includes a lot of new features. Attribute Routing ASP.NET Web API supports the same attribute routing system that's in ASP.NET MVC 5. You can read more about the Attribute Routing features in Web API in this article. OAuth 2.0 ASP.NET Web API picks up OAuth 2.0 support, using security middleware running on OWIN (discussed below). This is great for features like authenticated Single Page Applications. OData Improvements ASP.NET Web API now has full OData support. That required adding in some of the most powerful operators: $select, $expand, $batch and $value. You can read more about OData operator support in this article by Mike Wasson. Lots more There's a huge list of other features, including CORS (cross-origin request sharing), IHttpActionResult, IHttpRequestContext, and more. I think the best overview is in the release notes. OWIN and Katana I've written about OWIN and Katana recently. I'm a big fan. OWIN is the Open Web Interfaces for .NET. It's a spec, like HTML or HTTP, so you can't install OWIN. The benefit of OWIN is that it's a community specification, so anyone who implements it can plug into the ASP.NET stack, either as middleware or as a host. Katana is the Microsoft implementation of OWIN. It leverages OWIN to wire up things like authentication, handlers, modules, IIS hosting, etc., so ASP.NET can host OWIN components and Katana components can run in someone else's OWIN implementation. Howard Dierking just wrote a cool article in MSDN magazine describing Katana in depth: Getting Started with the Katana Project. He had an interesting example showing an OWIN based pipeline which leveraged SignalR, ASP.NET Web API and NancyFx components in the same stack. If this kind of thing makes sense to you, that's great. If it doesn't, don't worry, but keep an eye on it. You're going to see some cool things happen as a result of ASP.NET becoming more and more pluggable. Visual Studio Web Tools Okay, this stuff's just crazy. Visual Studio has been adding some nice web dev features over the past few years, but they've really cranked it up for this release. Visual Studio is by far my favorite code editor for all web files: CSS, HTML, JavaScript, and lots of popular libraries. Stop thinking of Visual Studio as a big editor that you only use to write back-end code. Stop editing HTML and CSS in Notepad (or Sublime, Notepad++, etc.). Visual Studio starts up in under 2 seconds on a modern computer with an SSD. Misspelling HTML attributes or your CSS classes or jQuery or Angular syntax is stupid. It doesn't make you a better developer, it makes you a silly person who wastes time. Browser Link Browser Link is a real-time, two-way connection between Visual Studio and all connected browsers. It's only attached when you're running locally, in debug, but it applies to any and all connected browser, including emulators. You may have seen demos that showed the browsers refreshing based on changes in the editor, and I'll agree that's pretty cool. But it's really just the start. It's a two-way connection, and it's built for extensiblity. That means you can write extensions that push information from your running application (in IE, Chrome, a mobile emulator, etc.) back to Visual Studio. Mads and team have showed off some demonstrations where they enabled edit mode in the browser which updated the source HTML back on the browser. It's also possible to look at how the rendered HTML performs, check for compatibility issues, watch for unused CSS classes, the sky's the limit. New HTML editor The previous HTML editor had a lot of old code that didn't allow for improvements. The team rewrote the HTML editor to take advantage of the new(ish) extensibility features in Visual Studio, which then allowed them to add in all kinds of features - things like CSS Class and ID IntelliSense (so you type style="" and get a list of classes and ID's for your project), smart indent based on how your document is formatted, JavaScript reference auto-sync, etc. Here's a 3 minute tour from Mads Kristensen. The previous HTML editor had a lot of old code that didn't allow for improvements. The team rewrote the HTML editor to take advantage of the new(ish) extensibility features in Visual Studio, which then allowed them to add in all kinds of features - things like CSS Class and ID IntelliSense (so you type style="" and get a list of classes and ID's for your project), smart indent based on how your document is formatted, JavaScript reference auto-sync, etc. Lots more Visual Studio web dev features That's just a sampling - there's a ton of great features for JavaScript editing, CSS editing, publishing, and Page Inspector (which shows real-time rendering of your page inside Visual Studio). Here are some more short videos showing those features. Lots, lots more Okay, that's just a summary, and it's still quite a bit. Head on over to http://asp.net/vnext for more information, and download Visual Studio 2013 now to get started!

  Read the article

• Upgrade Your Existing BI Publisher 11g (11.1.1.3) to 11.1.1.5

  - by Kan Nishida

  It’s already more than a month now since BI Publisher 11.1.1.5 was released at beginning of May. Have you already tried out many of the great new features? If you are already running on the first version of BI Publisher 11g (11.1.1.3) you might wonder how to upgrade the existing BI Publisher to the 11.1.1.5 version. There are two ways to do this, one is ‘Out-Place’ and another is ‘In-Place’. The ‘Out-Place’ would be quite simple. Basically you will need to install the whole BI or just BI Publisher standalone R11.1.1.5 at a different location then you can switch the catalog to the existing one so that all the reports will be there in the new 11.1.1.5 environment. But sometimes things are not that simple, you might have some custom applications or configuration on the original environment and you want to keep all of them with the upgraded environment. For such scenarios, there is the ‘In-Place’ upgrade, which overrides on top of the original environment only the parts relevant for BI and BI Publisher, and that’s what I’m going to talk about today. Here is the basic steps of the ‘In-Place’ upgrade. Upgrade WebLogic Server to 10.3.5 Upgrade BI System to 11.1.1.5 Upgrade Database Schema Re-register BI Components Upgrade FMW (Fusion Middleware) Configuration Upgrade BI Catalog There is a section that talks about this upgrade from 11.1.1.3 to 11.1.1.5 as part of the overall upgrade document. But I hope my blog post summarized it and made it simple for you to cover only what’s necessary. Upgrade Document: http://download.oracle.com/docs/cd/E21764_01/bi.1111/e16452/bi_plan.htm#BABECJJH Before You Start Stop BI System and Backup I can’t emphasize enough, but before you start PLEASE make sure you take a backup of the existing environments first. You want to stop all WebLogic Servers, Node Manager, OPMN, and OPMN-managed system components that are part of your Oracle BI domains. If you’re on Windows you can do this by simply selecting ‘Stop BI Services’ menu. Then backup the whole system. Upgrade WebLogic Server to 10.3.5 Download WebLogic Server 10.3.5 Upgrade Installer With BI 11.1.1.3 installation your WebLogic Server (WLS) is 10.3.3 and you need to upgrade this to 10.3.5 before upgrading the BI part. In order to upgrade you will need this 10.3.5 upgrade version of WLS, which you can download from our support web site (https://support.oracle.com) You can find the detail information about the installation and the patch numbers for the WLS upgrade installer on this document. Just for your short cut, if you are running on Windows or Linux (x86) here is the patch number for your platform. Windows 32 bit: 12395517: Linux: 12395517 Upgrade WebLogic Server 1. After unzip the downloaded file, launch wls1035_upgrade_win32.exe if you’re on Windows. 2. Accept all the default values and keep ‘Next’ till end, and start the upgrade. Once the upgrade process completes you’ll see the following window. Now let’s move to the BI upgrade. Upgrade BI Platform to 11.1.1.5 with Software Only Install Download BI 11.1.1.5 You can download the 11.1.1.5 version from our OTN page for your evaluation or development. For the production use it’s recommended to download from eDelivery. 1. Launch the installer by double click ‘setup.exe’ (for Windows) 2. Select ‘Software Only Install’ option 3. Select your original Oracle Home where you installed BI 11.1.1.3. 4. Click ‘Install’ button to start the installation. And now the software part of the BI has been upgraded to 11.1.1.5. Now let’s move to the database schema upgrade. Upgrade Database Schema with Patch Assistant You need to upgrade the BIPLATFORM and MDS Schemas. You can use the Patch Assistant utility to do this, and here is an example assuming you’ve created the schema with ‘DEV’ prefix, otherwise change it with yours accordingly. Upgrade BIPLATFORM schema (if you created this schema with DEV_ prev) psa.bat -dbConnectString localhost:1521:orcl -dbaUserName sys -schemaUserName DEV_BIPLATFORM Upgrade MDS schema (if you created this schema with DEV_ prev) psa.bat -dbConnectString localhost:1521:orcl -dbaUserName sys -schemaUserName DEV_MDS Re-register BI System components Now you need to re-register your BI system components such as BI Server, BI Presentation Server, etc to the Fusion Middleware system. You can do this by running ‘upgradenonj2eeapp.bat (or .sh)’ command, which can be found at %ORACLE_HOME%/opmn/bin. Before you run, you need to start the WLS Server and make sure your WLS environment is not locked. If it’s locked then you need to release the system from the Fusion Middleware console before you run the following command. Here is the syntax for the ‘upgradenonj2eeapp.bat (or .sh) command.  upgradenonj2eeapp.bat    -oracleInstance Instance_Home_Location    -adminHost WebLogic_Server_Host_Name    -adminPort administration_server_port_number    -adminUsername administration_server_user And here is an example: cd %BI_HOME%\opmn\bin upgradenonj2eeapp.bat -oracleInstance C:\biee11\instances\instance1 -adminHost localhost -adminPort 7001 -adminUsername weblogic Upgrade Fusion Middleware Configuration There are a couple things on the Fusion Middleware need to be upgraded for the BI system to work. Here is a list of the components to upgrade. Upgrade Shared Library (JRF) Upgrade Fusion Middleware Security (OPSS) Upgrade Code Grants Upgrade OWSM Policy Repository Before moving forward, you need to stop the WebLogic Server. Here is an example. cd %MW_HOME%user_projects\domains\bifoundation_domain\binstopWebLogic.cmd And, let’s start with ‘Upgrade Shared Library (JRF)’. Upgrade Shared Library (JRF) You can use updateJRF() WLST command to upgrade the shared libraries in your domain. Before you do this, you need to stop all running instances, Managed Servers, Administration Server, and Node Manager in the domain. Here is an example of the ‘upgradeJRF()’ command: cd %MW_HOME%\oracle_common\common\bin wlst.cmd upgradeJRF('C:/biee11/user_projects/domains/bifoundation_domain') Upgrade Fusion Middleware Security (OPSS) This step is to upgrade the Fusion Middleware security piece. You can use ‘upgradeOpss()’ WLST command. Here is a syntax for the command. upgradeOpss(jpsConfig="existing_jps_config_file", jaznData="system_jazn_data_file") The ‘existing jps-config.xml file can be found under %DOMAIN_HOME%/config/fmwconfig/jps-config.xml and the ‘system_jazn_data_file’ can be found under %MW_HOME%/oracle_common/modules/oracle.jps_11.1.1/domain_config/system-jazn-data.xml. And here is an example: cd %MW_HOME%\oracle_common\common\bin wlst.cmd upgradeOpss(jpsConfig="c:/biee11/user_projects/domains/bifoundation_domain/config/fmwconfig/jps-config.xml", jaznData="c:/biee11/oracle_common/modules/oracle.jps_11.1.1/domain_config/system-jazn-data.xml") exit() Upgrade Code Grants for Oracle BI Domain And this is the last step for the Fusion Middleware platform upgrade task. You need to run this python script ‘bi-upgrade.py‘ script to configure the code grants necessary to ensure that SSL works correctly for Oracle BI. However, even if you don’t use SSL, you still need to run this script. And if you have multiple BI domains (Enterprise deployment) then you need to run this on each domain. Here is an example: cd %MW_HOME%\oracle_common\common\bin wlst c:\biee11\Oracle_BI1\bin\bi-upgrade.py --bioraclehome c:\biee11\Oracle_BI1 --domainhome c:\biee11\user_projects\domains\bifoundation_domain Upgrade OWSM Policy Repository This is to upgrade OWSM (Oracle Web Service Manager) policy repository, you can use WLST command ‘upgradeWSMPolicyRepository()’. In order to run this command you need to have your WebLogic Server up-and-running. Here is an example. cd %MW_HOME%user_projects\domains\bifoundation_domain\binstopWebLogic.cmd cd %MW_HOME%\oracle_common\common\bin wlst.cmd connect ('weblogic','welcome1','t3://localhost:7001') upgradeWSMPolicyRepository() exit() Upgrade BI Catalogs This step is required only when you have your BI Publisher integrated with BIEE. If your BI Publisher is deployed as a standalone then you don’t need to follow this step. Now finally, you can upgrade the BI catalog. This won’t upgrade your BI Publisher reports themselves, but it just upgrades some attributes information inside the catalog. Before you do this upgrade, make sure the BI system components are not running. You can check the status by the command below. opmnctl status You can do the upgrade by updating a configuration file ‘instanceconfig.xml’, which can be found at %BI_HOME%\instances\instance1\config\coreapplication_obips1, and change the value of ‘UpgradeAndExit’ to be ‘true’. Here is an example: <ps:Catalog xmlns:ps="oracle.bi.presentation.services/config/v1.1"> <ps:UpgradeAndExit>true</ps:UpgradeAndExit> </ps:Catalog> After you made the change and save the file, you need to start the BI Presentation Server. This time you want to start only the BI Presentation Server instead of starting all the servers. You can use ‘opmnctl’ to do so, and here is an example. cd %ORACLE_INSTANCE%\bin opmnctl startproc ias-component=coreapplication_obips1 This would upgrade your BI Catalog to be 11.1.1.5. After the catalog is updated, you can stop the BI Presentation Server so that you can modify the instanceconfig.xml file again to revert the upgradeAndExit value back to ‘false’. Start Explore BI Publisher 11.1.1.5 After all the above steps, you can start all the BI Services, access to the same URL, now you have your BI Publisher and/or BI 11.1.1.5 in your hands. Have fun exploring all the new features of R11.1.1.5!

  Read the article

• CodePlex Daily Summary for Monday, March 01, 2010

  CodePlex Daily Summary for Monday, March 01, 2010New ProjectsActiveWorlds World Server Admin PowerShell SnapIn: The purpose of this PowerShell SnapIn is to provide a set of tools to administer the world server from PowerShell. It leverages the ActiveWorlds S...AWS SimpleDB Browser: A basic GUI browser tool for inspection and querying of a SimpleDB.Desktop Dimmer: A simple application for dimming the desktop around windows, videos, or other media.Disk Defuzzer: Compare chaos of files and folders with customizable SQL queries. This little application scans files in any two folders, generates data in an A...Dynamic Configuration: Dynamic configuration is a (very) small library to provide an API compatible replacement for the System.Configuration.ConfigurationManager class so...Expression Encoder 3 Visual Basic Samples: Visual Basic Sample code that calls the Expression Encoder 3 object model.Extended Character Keyboard: An lightweight onscreen keyboard that allows you to enter special characters like "á" and "û". Also supports adding of 7 custom buttons.FileHasher: This project provides a simple tool for generating and verifying file hashes. I created this to help the QA team I work with. The project is all C#...Fluent Assertions: Fluent interface for writing more natural specifying assertions with more clarity than the traditional assertion syntax such as offered by MSTest, ...Foursquare BlogEngine Widget: A Basic Widget for BlogEngine which displays the last foursquare Check-insGraffiti CMS Events Plugin: Plugin for Graffiti CMS that allows creating Event posts and rendering an Event CalendarHeadCounter: HeadCounter is a raid attendance and loot tracking application for World of Warcraft.HRM Core (QL Nhan Su): This is software about Human Resource Management in Viet Nam ------------ Đây là phần mềm Quản lý nhân sự tiền lương ở Việt Nam (Nghiệp vụ ở Việt Nam)IronPython Silverlight Sharpdevelop Template: This IronPython Silverlight SharpDevelop Template makes it easier for you to make Silverlight applications in IronPython with Sharpdevelop.kingbox: my test code for study vs 2005link_attraente: Projeto Conclusão de CursoORMSharp.Net: ORMSharp.Net https://code.google.com/p/ormsharp/ http://www.sqlite.org/ http://sqlite.phxsoftware.com/ http://sourceforge.net/projects/sqlite-dotnet2/Orz framework: Orz framework is more like a helpful library, if you are develop with DotNet framework 3.0, it will be very useful to you. Orz framework encapsul...OTManager: OTManagerSharePoint URL Ping Tool: The Url Ping Tool is a farm feature in SharePoint that provide additional performance and tracing information that can be used to troubleshoot issu...SunShine: SunShine ProjectToolSuite.ValidationExpression: assembly with regular expression for the RegularExpressionValidator controlTwitual Studio: A Visual Studio 2010 based Twitter client. Now you have one less reason for pressing Alt+Tab. Plus you still look like you're working!Velocity Hosting Tool: A program designed to aid a HT Velocity host in hosting and recording tournaments.Watermarker: Adds watermark on pictures to prevent copy. Icon taken from PICOL. Can work with packs of images.Zack's Fiasco - ASP.NET Script Includer: Script includer to * include scripts (JS or CSS) once and only once. * include the correct format by differentiating between release and build. Th...New ReleasesAll-In-One Code Framework: All-In-One Code Framework 2010-02-28: Improved and Newly Added Examples:For an up-to-date list, please refer to All-In-One Code Framework Sample Catalog. Samples for ASP.NET Name ...All-In-One Code Framework (简体中文): All-In-One Code Framework 2010-02-28: Improved and Newly Added Examples:For an up-to-date list, please refer to All-In-One Code Framework Sample Catalog. Latest Download Link: http://c...AWS SimpleDB Browser: SimpleDbBrowser.zip Initial Release: The initial release of the SimpleDbBrowser. Unzip the file in the archive and place them all in a folder, then run the .exe. No installer is used...BattLineSvc: V1: First release of BattLineSvcCC.Votd Screen Saver: CC.Votd 1.0.10.301: More bug fixes and minor enhancements. Note: Only download the (Screen Saver) version if you plan to manually install. For most users the (Install...Dynamic Configuration: DynamicConfiguration Release 1: Dynamic Configuration DLL release.eIDPT - Cartão de Cidadão .NET Wrapper: EIDPT VB6 Demo Program: Cartão de Cidadão Middleware Application installation (v1.21 or 1.22) is required for proper use of the eID Lib.eIDPT - Cartão de Cidadão .NET Wrapper: eIDPT VB6 Demo Program Source: Cartão de Cidadão Middleware Application installation (v1.21 or 1.22) is required for proper use of the eID Lib.ESPEHA: Espeha 10: 1. Help available on F1 and via context menu '?' 2. Width of categiries view is preserved througb app starts 3. Drag'nd'drop for tasks view allows ...Extended Character Keyboard: OnscreenSCK Beta V1.0: OnscreenSCK Beta Version 1.0Extended Character Keyboard: OnscreenSCK Beta V1.0 Source: OnscreenSCK Beta Version 1.0 Source CodeFileHasher: Console Version v 0.5: This release provides a very basic and minimal command-line utility for generating and validating file hashes. The supported command-line paramete...Furcadia Framework for Third Party Programs: 0.2.3 Epic Wrench: Warning: Untested on Linux.FurcadiaLib\Net\NetProxy.cs: Fixed a bug I made before update. FurcadiaFramework_Example\Demo\IDemo.cs: Ignore me. F...Graffiti CMS Events Plugin: Version 1.0: Initial Release of Events PluginHeadCounter: HeadCounter 1.2.3 'Razorgore': Added "Raider Post" feature for posting details of a particular raider. Added Default Period option to allow selection of Short, Long or Lifetime...Home Access Plus+: v3.0.0.0: Version 3.0.0.0 Release Change Log: Reconfiguration of the web.config Ability to add additional links to homepage via web.config Ability to add...Home Access Plus+: v3.0.1.0: Version 3.0.1.0 Release Change Log: Fixed problem with moving File Changes: ~/bin/chs extranet.dll ~/bin/chs extranet.pdbHome Access Plus+: v3.0.2.0: Version 3.0.2.0 Release Change Log: Fixed problem with stylesheet File Changes: ~/chs.masterHRM Core (QL Nhan Su): HRMCore_src: Source of HRMCoreIRC4N00bz: IRC4N00bz v1.0.0.2: There wasn't much updated this weekend. I updated 2 'raw' events. One is all raw messages and the other is events that arn't caught in the dll. ...IronPython Silverlight Sharpdevelop Template: Version 1 Template: Just unzip it into the Sharpdevelop python templates folder For example: C:\Program Files\SharpDevelop\3.0\AddIns\AddIns\BackendBindings\PythonBi...MDownloader: MDownloader-0.15.4.56156: Fixed handling exceptions; previous handling could lead to freezing items state; Fixed validating uploading.com links;OTManager: Activity Log: 2010.02.28 >> Thread Reopened 2010.02.28 >> Re-organized WBD Features/WMBD Features 2010.02.28 >> Project status is active againPicasa Downloader: PicasaDownloader (41175): NOTE: The previous release was accidently the same as the one before that (forgot to rebuild the installer). Changelog: Fixed workitem 10296 (Sav...PicNet Html Table Filter: Version 2.0: Testing w/ JQuery 1.3.2Program Scheduler: Program Scheduler 1.1.4: Release Note: *Bug fix : If the log window is docked and user moves the log window , main window will move too. *Added menu to log window to clear...QueryToGrid Module for DotNetNuke®: QueryToGrid Module version 01.00.00: This is the initial release of this module. Remember... This is just a proof of concept to add AJAX functionality to your DotNetNuke modules.Rainweaver Framework: February 2010 Release: Code drop including an Alpha release of the Entity System. See more information in the Documentation page.RapidWebDev - .NET Enterprise Software Development Infrastructure: ProductManagement Quick Sample 0.1: This is a sample product management application to demonstrate how to develop enterprise software in RapidWebDev. The glossary of the system are ro...Team Foundation Server Revision Labeller for CruiseControl.NET: TFS Labeller for CruiseControl.NET - TFS 2008: ReleaseFirst release of the Team Foundation Server Labeller for CruiseControl.NET. This specific version is bound to TFS 2008 DLLs.ToolSuite.ValidationExpression: 01.00.01.000: first release of the time validation class; the assembly file is ready to use, the documentation ist not complete;VCC: Latest build, v2.1.30228.0: Automatic drop of latest buildWatchersNET CKEditor™ Provider for DotNetNuke: CKEditor Provider 1.7.00: Whats New FileBrowser: Non Admin Users will only see a User Sub folder (..\Portals\0\userfiles\UserName) CKFinder: Non Admin Users will only see ...Watermarker: Watermarker: first public version. can build watermark only in left top corner on one image at once.While You Were Away - WPF Screensaver: Initial Release: This is the code released when the article went live.Most Popular ProjectsMetaSharpRawrWBFS ManagerAJAX Control ToolkitMicrosoft SQL Server Product Samples: DatabaseSilverlight ToolkitWindows Presentation Foundation (WPF)Microsoft SQL Server Community & SamplesASP.NETDotNetNuke® Community EditionMost Active ProjectsRawrBlogEngine.NETMapWindow GISCommon Context Adapterspatterns & practices – Enterprise LibrarySharpMap - Geospatial Application Framework for the CLRSLARToolkit - Silverlight Augmented Reality ToolkitDiffPlex - a .NET Diff GeneratorRapid Entity Framework. (ORM). CTP 2jQuery Library for SharePoint Web Services

  Read the article

• A Taxonomy of Numerical Methods v1

  - by JoshReuben

  Numerical Analysis – When, What, (but not how) Once you understand the Math & know C++, Numerical Methods are basically blocks of iterative & conditional math code. I found the real trick was seeing the forest for the trees – knowing which method to use for which situation. Its pretty easy to get lost in the details – so I’ve tried to organize these methods in a way that I can quickly look this up. I’ve included links to detailed explanations and to C++ code examples. I’ve tried to classify Numerical methods in the following broad categories: Solving Systems of Linear Equations Solving Non-Linear Equations Iteratively Interpolation Curve Fitting Optimization Numerical Differentiation & Integration Solving ODEs Boundary Problems Solving EigenValue problems Enjoy – I did ! Solving Systems of Linear Equations Overview Solve sets of algebraic equations with x unknowns The set is commonly in matrix form Gauss-Jordan Elimination http://en.wikipedia.org/wiki/Gauss%E2%80%93Jordan_elimination C++: http://www.codekeep.net/snippets/623f1923-e03c-4636-8c92-c9dc7aa0d3c0.aspx Produces solution of the equations & the coefficient matrix Efficient, stable 2 steps: · Forward Elimination – matrix decomposition: reduce set to triangular form (0s below the diagonal) or row echelon form. If degenerate, then there is no solution · Backward Elimination –write the original matrix as the product of ints inverse matrix & its reduced row-echelon matrix à reduce set to row canonical form & use back-substitution to find the solution to the set Elementary ops for matrix decomposition: · Row multiplication · Row switching · Add multiples of rows to other rows Use pivoting to ensure rows are ordered for achieving triangular form LU Decomposition http://en.wikipedia.org/wiki/LU_decomposition C++: http://ganeshtiwaridotcomdotnp.blogspot.co.il/2009/12/c-c-code-lu-decomposition-for-solving.html Represent the matrix as a product of lower & upper triangular matrices A modified version of GJ Elimination Advantage – can easily apply forward & backward elimination to solve triangular matrices Techniques: · Doolittle Method – sets the L matrix diagonal to unity · Crout Method - sets the U matrix diagonal to unity Note: both the L & U matrices share the same unity diagonal & can be stored compactly in the same matrix Gauss-Seidel Iteration http://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_method C++: http://www.nr.com/forum/showthread.php?t=722 Transform the linear set of equations into a single equation & then use numerical integration (as integration formulas have Sums, it is implemented iteratively). an optimization of Gauss-Jacobi: 1.5 times faster, requires 0.25 iterations to achieve the same tolerance Solving Non-Linear Equations Iteratively find roots of polynomials – there may be 0, 1 or n solutions for an n order polynomial use iterative techniques Iterative methods · used when there are no known analytical techniques · Requires set functions to be continuous & differentiable · Requires an initial seed value – choice is critical to convergence à conduct multiple runs with different starting points & then select best result · Systematic - iterate until diminishing returns, tolerance or max iteration conditions are met · bracketing techniques will always yield convergent solutions, non-bracketing methods may fail to converge Incremental method if a nonlinear function has opposite signs at 2 ends of a small interval x1 & x2, then there is likely to be a solution in their interval – solutions are detected by evaluating a function over interval steps, for a change in sign, adjusting the step size dynamically. Limitations – can miss closely spaced solutions in large intervals, cannot detect degenerate (coinciding) solutions, limited to functions that cross the x-axis, gives false positives for singularities Fixed point method http://en.wikipedia.org/wiki/Fixed-point_iteration C++: http://books.google.co.il/books?id=weYj75E_t6MC&pg=PA79&lpg=PA79&dq=fixed+point+method++c%2B%2B&source=bl&ots=LQ-5P_taoC&sig=lENUUIYBK53tZtTwNfHLy5PEWDk&hl=en&sa=X&ei=wezDUPW1J5DptQaMsIHQCw&redir_esc=y#v=onepage&q=fixed%20point%20method%20%20c%2B%2B&f=false Algebraically rearrange a solution to isolate a variable then apply incremental method Bisection method http://en.wikipedia.org/wiki/Bisection_method C++: http://numericalcomputing.wordpress.com/category/algorithms/ Bracketed - Select an initial interval, keep bisecting it ad midpoint into sub-intervals and then apply incremental method on smaller & smaller intervals – zoom in Adv: unaffected by function gradient à reliable Disadv: slow convergence False Position Method http://en.wikipedia.org/wiki/False_position_method C++: http://www.dreamincode.net/forums/topic/126100-bisection-and-false-position-methods/ Bracketed - Select an initial interval , & use the relative value of function at interval end points to select next sub-intervals (estimate how far between the end points the solution might be & subdivide based on this) Newton-Raphson method http://en.wikipedia.org/wiki/Newton's_method C++: http://www-users.cselabs.umn.edu/classes/Summer-2012/csci1113/index.php?page=./newt3 Also known as Newton's method Convenient, efficient Not bracketed – only a single initial guess is required to start iteration – requires an analytical expression for the first derivative of the function as input. Evaluates the function & its derivative at each step. Can be extended to the Newton MutiRoot method for solving multiple roots Can be easily applied to an of n-coupled set of non-linear equations – conduct a Taylor Series expansion of a function, dropping terms of order n, rewrite as a Jacobian matrix of PDs & convert to simultaneous linear equations !!! Secant Method http://en.wikipedia.org/wiki/Secant_method C++: http://forum.vcoderz.com/showthread.php?p=205230 Unlike N-R, can estimate first derivative from an initial interval (does not require root to be bracketed) instead of inputting it Since derivative is approximated, may converge slower. Is fast in practice as it does not have to evaluate the derivative at each step. Similar implementation to False Positive method Birge-Vieta Method http://mat.iitm.ac.in/home/sryedida/public_html/caimna/transcendental/polynomial%20methods/bv%20method.html C++: http://books.google.co.il/books?id=cL1boM2uyQwC&pg=SA3-PA51&lpg=SA3-PA51&dq=Birge-Vieta+Method+c%2B%2B&source=bl&ots=QZmnDTK3rC&sig=BPNcHHbpR_DKVoZXrLi4nVXD-gg&hl=en&sa=X&ei=R-_DUK2iNIjzsgbE5ID4Dg&redir_esc=y#v=onepage&q=Birge-Vieta%20Method%20c%2B%2B&f=false combines Horner's method of polynomial evaluation (transforming into lesser degree polynomials that are more computationally efficient to process) with Newton-Raphson to provide a computational speed-up Interpolation Overview Construct new data points for as close as possible fit within range of a discrete set of known points (that were obtained via sampling, experimentation) Use Taylor Series Expansion of a function f(x) around a specific value for x Linear Interpolation http://en.wikipedia.org/wiki/Linear_interpolation C++: http://www.hamaluik.com/?p=289 Straight line between 2 points à concatenate interpolants between each pair of data points Bilinear Interpolation http://en.wikipedia.org/wiki/Bilinear_interpolation C++: http://supercomputingblog.com/graphics/coding-bilinear-interpolation/2/ Extension of the linear function for interpolating functions of 2 variables – perform linear interpolation first in 1 direction, then in another. Used in image processing – e.g. texture mapping filter. Uses 4 vertices to interpolate a value within a unit cell. Lagrange Interpolation http://en.wikipedia.org/wiki/Lagrange_polynomial C++: http://www.codecogs.com/code/maths/approximation/interpolation/lagrange.php For polynomials Requires recomputation for all terms for each distinct x value – can only be applied for small number of nodes Numerically unstable Barycentric Interpolation http://epubs.siam.org/doi/pdf/10.1137/S0036144502417715 C++: http://www.gamedev.net/topic/621445-barycentric-coordinates-c-code-check/ Rearrange the terms in the equation of the Legrange interpolation by defining weight functions that are independent of the interpolated value of x Newton Divided Difference Interpolation http://en.wikipedia.org/wiki/Newton_polynomial C++: http://jee-appy.blogspot.co.il/2011/12/newton-divided-difference-interpolation.html Hermite Divided Differences: Interpolation polynomial approximation for a given set of data points in the NR form - divided differences are used to approximately calculate the various differences. For a given set of 3 data points , fit a quadratic interpolant through the data Bracketed functions allow Newton divided differences to be calculated recursively Difference table Cubic Spline Interpolation http://en.wikipedia.org/wiki/Spline_interpolation C++: https://www.marcusbannerman.co.uk/index.php/home/latestarticles/42-articles/96-cubic-spline-class.html Spline is a piecewise polynomial Provides smoothness – for interpolations with significantly varying data Use weighted coefficients to bend the function to be smooth & its 1st & 2nd derivatives are continuous through the edge points in the interval Curve Fitting A generalization of interpolating whereby given data points may contain noise à the curve does not necessarily pass through all the points Least Squares Fit http://en.wikipedia.org/wiki/Least_squares C++: http://www.ccas.ru/mmes/educat/lab04k/02/least-squares.c Residual – difference between observed value & expected value Model function is often chosen as a linear combination of the specified functions Determines: A) The model instance in which the sum of squared residuals has the least value B) param values for which model best fits data Straight Line Fit Linear correlation between independent variable and dependent variable Linear Regression http://en.wikipedia.org/wiki/Linear_regression C++: http://www.oocities.org/david_swaim/cpp/linregc.htm Special case of statistically exact extrapolation Leverage least squares Given a basis function, the sum of the residuals is determined and the corresponding gradient equation is expressed as a set of normal linear equations in matrix form that can be solved (e.g. using LU Decomposition) Can be weighted - Drop the assumption that all errors have the same significance –-> confidence of accuracy is different for each data point. Fit the function closer to points with higher weights Polynomial Fit - use a polynomial basis function Moving Average http://en.wikipedia.org/wiki/Moving_average C++: http://www.codeproject.com/Articles/17860/A-Simple-Moving-Average-Algorithm Used for smoothing (cancel fluctuations to highlight longer-term trends & cycles), time series data analysis, signal processing filters Replace each data point with average of neighbors. Can be simple (SMA), weighted (WMA), exponential (EMA). Lags behind latest data points – extra weight can be given to more recent data points. Weights can decrease arithmetically or exponentially according to distance from point. Parameters: smoothing factor, period, weight basis Optimization Overview Given function with multiple variables, find Min (or max by minimizing –f(x)) Iterative approach Efficient, but not necessarily reliable Conditions: noisy data, constraints, non-linear models Detection via sign of first derivative - Derivative of saddle points will be 0 Local minima Bisection method Similar method for finding a root for a non-linear equation Start with an interval that contains a minimum Golden Search method http://en.wikipedia.org/wiki/Golden_section_search C++: http://www.codecogs.com/code/maths/optimization/golden.php Bisect intervals according to golden ratio 0.618.. Achieves reduction by evaluating a single function instead of 2 Newton-Raphson Method Brent method http://en.wikipedia.org/wiki/Brent's_method C++: http://people.sc.fsu.edu/~jburkardt/cpp_src/brent/brent.cpp Based on quadratic or parabolic interpolation – if the function is smooth & parabolic near to the minimum, then a parabola fitted through any 3 points should approximate the minima – fails when the 3 points are collinear , in which case the denominator is 0 Simplex Method http://en.wikipedia.org/wiki/Simplex_algorithm C++: http://www.codeguru.com/cpp/article.php/c17505/Simplex-Optimization-Algorithm-and-Implemetation-in-C-Programming.htm Find the global minima of any multi-variable function Direct search – no derivatives required At each step it maintains a non-degenerative simplex – a convex hull of n+1 vertices. Obtains the minimum for a function with n variables by evaluating the function at n-1 points, iteratively replacing the point of worst result with the point of best result, shrinking the multidimensional simplex around the best point. Point replacement involves expanding & contracting the simplex near the worst value point to determine a better replacement point Oscillation can be avoided by choosing the 2nd worst result Restart if it gets stuck Parameters: contraction & expansion factors Simulated Annealing http://en.wikipedia.org/wiki/Simulated_annealing C++: http://code.google.com/p/cppsimulatedannealing/ Analogy to heating & cooling metal to strengthen its structure Stochastic method – apply random permutation search for global minima - Avoid entrapment in local minima via hill climbing Heating schedule - Annealing schedule params: temperature, iterations at each temp, temperature delta Cooling schedule – can be linear, step-wise or exponential Differential Evolution http://en.wikipedia.org/wiki/Differential_evolution C++: http://www.amichel.com/de/doc/html/ More advanced stochastic methods analogous to biological processes: Genetic algorithms, evolution strategies Parallel direct search method against multiple discrete or continuous variables Initial population of variable vectors chosen randomly – if weighted difference vector of 2 vectors yields a lower objective function value then it replaces the comparison vector Many params: #parents, #variables, step size, crossover constant etc Convergence is slow – many more function evaluations than simulated annealing Numerical Differentiation Overview 2 approaches to finite difference methods: · A) approximate function via polynomial interpolation then differentiate · B) Taylor series approximation – additionally provides error estimate Finite Difference methods http://en.wikipedia.org/wiki/Finite_difference_method C++: http://www.wpi.edu/Pubs/ETD/Available/etd-051807-164436/unrestricted/EAMPADU.pdf Find differences between high order derivative values - Approximate differential equations by finite differences at evenly spaced data points Based on forward & backward Taylor series expansion of f(x) about x plus or minus multiples of delta h. Forward / backward difference - the sums of the series contains even derivatives and the difference of the series contains odd derivatives – coupled equations that can be solved. Provide an approximation of the derivative within a O(h^2) accuracy There is also central difference & extended central difference which has a O(h^4) accuracy Richardson Extrapolation http://en.wikipedia.org/wiki/Richardson_extrapolation C++: http://mathscoding.blogspot.co.il/2012/02/introduction-richardson-extrapolation.html A sequence acceleration method applied to finite differences Fast convergence, high accuracy O(h^4) Derivatives via Interpolation Cannot apply Finite Difference method to discrete data points at uneven intervals – so need to approximate the derivative of f(x) using the derivative of the interpolant via 3 point Lagrange Interpolation Note: the higher the order of the derivative, the lower the approximation precision Numerical Integration Estimate finite & infinite integrals of functions More accurate procedure than numerical differentiation Use when it is not possible to obtain an integral of a function analytically or when the function is not given, only the data points are Newton Cotes Methods http://en.wikipedia.org/wiki/Newton%E2%80%93Cotes_formulas C++: http://www.siafoo.net/snippet/324 For equally spaced data points Computationally easy – based on local interpolation of n rectangular strip areas that is piecewise fitted to a polynomial to get the sum total area Evaluate the integrand at n+1 evenly spaced points – approximate definite integral by Sum Weights are derived from Lagrange Basis polynomials Leverage Trapezoidal Rule for default 2nd formulas, Simpson 1/3 Rule for substituting 3 point formulas, Simpson 3/8 Rule for 4 point formulas. For 4 point formulas use Bodes Rule. Higher orders obtain more accurate results Trapezoidal Rule uses simple area, Simpsons Rule replaces the integrand f(x) with a quadratic polynomial p(x) that uses the same values as f(x) for its end points, but adds a midpoint Romberg Integration http://en.wikipedia.org/wiki/Romberg's_method C++: http://code.google.com/p/romberg-integration/downloads/detail?name=romberg.cpp&can=2&q= Combines trapezoidal rule with Richardson Extrapolation Evaluates the integrand at equally spaced points The integrand must have continuous derivatives Each R(n,m) extrapolation uses a higher order integrand polynomial replacement rule (zeroth starts with trapezoidal) à a lower triangular matrix set of equation coefficients where the bottom right term has the most accurate approximation. The process continues until the difference between 2 successive diagonal terms becomes sufficiently small. Gaussian Quadrature http://en.wikipedia.org/wiki/Gaussian_quadrature C++: http://www.alglib.net/integration/gaussianquadratures.php Data points are chosen to yield best possible accuracy – requires fewer evaluations Ability to handle singularities, functions that are difficult to evaluate The integrand can include a weighting function determined by a set of orthogonal polynomials. Points & weights are selected so that the integrand yields the exact integral if f(x) is a polynomial of degree <= 2n+1 Techniques (basically different weighting functions): · Gauss-Legendre Integration w(x)=1 · Gauss-Laguerre Integration w(x)=e^-x · Gauss-Hermite Integration w(x)=e^-x^2 · Gauss-Chebyshev Integration w(x)= 1 / Sqrt(1-x^2) Solving ODEs Use when high order differential equations cannot be solved analytically Evaluated under boundary conditions RK for systems – a high order differential equation can always be transformed into a coupled first order system of equations Euler method http://en.wikipedia.org/wiki/Euler_method C++: http://rosettacode.org/wiki/Euler_method First order Runge–Kutta method. Simple recursive method – given an initial value, calculate derivative deltas. Unstable & not very accurate (O(h) error) – not used in practice A first-order method - the local error (truncation error per step) is proportional to the square of the step size, and the global error (error at a given time) is proportional to the step size In evolving solution between data points xn & xn+1, only evaluates derivatives at beginning of interval xn à asymmetric at boundaries Higher order Runge Kutta http://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods C++: http://www.dreamincode.net/code/snippet1441.htm 2nd & 4th order RK - Introduces parameterized midpoints for more symmetric solutions à accuracy at higher computational cost Adaptive RK – RK-Fehlberg – estimate the truncation at each integration step & automatically adjust the step size to keep error within prescribed limits. At each step 2 approximations are compared – if in disagreement to a specific accuracy, the step size is reduced Boundary Value Problems Where solution of differential equations are located at 2 different values of the independent variable x à more difficult, because cannot just start at point of initial value – there may not be enough starting conditions available at the end points to produce a unique solution An n-order equation will require n boundary conditions – need to determine the missing n-1 conditions which cause the given conditions at the other boundary to be satisfied Shooting Method http://en.wikipedia.org/wiki/Shooting_method C++: http://ganeshtiwaridotcomdotnp.blogspot.co.il/2009/12/c-c-code-shooting-method-for-solving.html Iteratively guess the missing values for one end & integrate, then inspect the discrepancy with the boundary values of the other end to adjust the estimate Given the starting boundary values u1 & u2 which contain the root u, solve u given the false position method (solving the differential equation as an initial value problem via 4th order RK), then use u to solve the differential equations. Finite Difference Method For linear & non-linear systems Higher order derivatives require more computational steps – some combinations for boundary conditions may not work though Improve the accuracy by increasing the number of mesh points Solving EigenValue Problems An eigenvalue can substitute a matrix when doing matrix multiplication à convert matrix multiplication into a polynomial EigenValue For a given set of equations in matrix form, determine what are the solution eigenvalue & eigenvectors Similar Matrices - have same eigenvalues. Use orthogonal similarity transforms to reduce a matrix to diagonal form from which eigenvalue(s) & eigenvectors can be computed iteratively Jacobi method http://en.wikipedia.org/wiki/Jacobi_method C++: http://people.sc.fsu.edu/~jburkardt/classes/acs2_2008/openmp/jacobi/jacobi.html Robust but Computationally intense – use for small matrices < 10x10 Power Iteration http://en.wikipedia.org/wiki/Power_iteration For any given real symmetric matrix, generate the largest single eigenvalue & its eigenvectors Simplest method – does not compute matrix decomposition à suitable for large, sparse matrices Inverse Iteration Variation of power iteration method – generates the smallest eigenvalue from the inverse matrix Rayleigh Method http://en.wikipedia.org/wiki/Rayleigh's_method_of_dimensional_analysis Variation of power iteration method Rayleigh Quotient Method Variation of inverse iteration method Matrix Tri-diagonalization Method Use householder algorithm to reduce an NxN symmetric matrix to a tridiagonal real symmetric matrix vua N-2 orthogonal transforms     Whats Next Outside of Numerical Methods there are lots of different types of algorithms that I’ve learned over the decades: Data Mining – (I covered this briefly in a previous post: http://geekswithblogs.net/JoshReuben/archive/2007/12/31/ssas-dm-algorithms.aspx ) Search & Sort Routing Problem Solving Logical Theorem Proving Planning Probabilistic Reasoning Machine Learning Solvers (eg MIP) Bioinformatics (Sequence Alignment, Protein Folding) Quant Finance (I read Wilmott’s books – interesting) Sooner or later, I’ll cover the above topics as well.

  Read the article

< Previous Page | 748 749 750 751 752 753 754 755 756  | Next Page >