I am looking for some well known algorithms that can be considered while handling very large amount of data.(Edit- By large amount of data I refer to records in a database excluding blobs). These algorithms if not in totality but in parts may be used in big web applications like Twitter, Last.fm , Amazon ,etc.
Specifically, I'm looking for names or links to such algorithms. My primary interest lies in developing a very deep understanding on working with large database records and writing efficient code for working with the same.
Hi, I'm looking for a good way to package a Python application that is going to be deployed on a Debian server.
The application itself depends on some modules which are not included in base Debian repository, although they might be in the future. This creates some problems... I depend on some patches to those modules. If the original module gets installed one day, the application will break. However if I install everything I need in a virtualenv just for that application, I lose the ability to upgrade Python itself (in case of security updates).
The third option would be to rename my fork of the upstream module and just treat it as a completely separate one. But that would mean changing the code (not much work, but it wouldn't be that clean / universal anymore).
Are there any other options that I missed? Are there any pros / cons I didn't see in the solutions above?
I am reading Domain-Driven Design by Evans and I am at the part discussing the layered architecture. I just realized that application and domain layers are different and should be separate. In the project I am working on, they are kind of blended and I can't tell the difference until I read the book (and I can't say it's very clear to me now), really.
My questions, since both of them concerns the logic of the application and are supposed to be clean of technical and presentation aspects, what are the advantages of drawing a boundary these two?
.NET has this concept of Application Domains which from what I understand can be used to load an assembly into memory. I've done some research on Application Domains as well as go to my local book store for some additional knowledge on this subject matter but it seems very scarce.
All I know that I can do with Application Domains is to load assemblies in memory and I can unload them when I want.
What are the capabilities other that I have mentioned of Application Domains? Do Threads respect Application Domains boundaries? Are there any drawbacks from loading Assemblies in different Application Domains other than the main Application Domains beyond performance of communication?
Links to resources that discuss Application Domains would be nice as well. I've already checked out MSDN which doesn't have that much information about them.
Where can I get good resources for learning cross platform application development in plain ISO/ANSI C? I have cygwin and eclipse cdt with me to start in my win7 pc. I just need a couple of good resources containing all the best practices and techniques to write good and robust and scalable cross platform application. I am totally new to this cross platform business, no prior idea. Want to learn it in a proper way from the very beginning. Please help me out here.
Is it possible to use ASP.NET Dynamic Data with SubSonic 3 in-place of Linq to SQL classes or the Entity Framework? MetaModel.RegisterContext() throws an exception if you use the context class that SubSonic generates. I thought I remembered coming across a SubSonic/Dynamic Data example back before SubSonic 3 was released but I can't find it now. Has anyone been able to get this to work?
IT security organizations are spending too much on data protection for compliance but not enough on securing key trade secrets, the real crown jewels of corporate data.
The blogosphere has a number of articles on the topic of AngularJS app structuring guidelines such as these (and others):
http://www.johnpapa.net/angular-app-structuring-guidelines/
http://codingsmackdown.tv/blog/2013/04/19/angularjs-modules-for-great-justice/
http://danorlando.com/angularjs-architecture-understanding-modules/
http://henriquat.re/modularizing-angularjs/modularizing-angular-applications/modularizing-angular-applications.html
However, one scenario I have yet to come across for guidelines and best practices is the case where you have a large web application containing multiple "mini-spa" apps, and the mini-spa apps all share a certain amount of code.
I am not referring to the case of trying to have multiple ng-app declarations on the same page; rather, I mean different sections of a large site that have their own, unique ng-app declaration.
As Scott Allen writes in his OdeToCode blog:
One scenario I haven't found addressed very well is the scenario where
multiple apps exist in the same greater web application and require
some shared code on the client.
Are there any recommended approaches to take, pitfalls to avoid, or good sample structures of this scenario that you can point to?
I'm getting random crashes when creating an inferred mapping model (with Core Data's lightweight migration) within my application. By the way, I have to do it programmatically in my application while it is running.
This is how I create this model (after I have made proper currentModel and newModel objects, of course):
NSMappingModel *mappingModel = [NSMappingModel inferredMappingModelForSourceModel:currentModel destinationModel:newModel error:&error];
The problem is this: This method is crashing randomly. When it works, it works just fine without issues. But when it crashes, it crashes my application (instead of returning nil to signify that the method failed, as it should). By randomly, I mean that sometimes it happens and sometimes not. It is unpredictable.
Now, here is the deal: I'm running this method in another thread. More precisely, it is located inside a block that is passed via GCD to run on the global main queue. I need to do this for my UI to appear crisp to the user, i.e. so that I can display a progress indicator while the work is underway.
The strange thing seems to be that if I remove the GCD stuff and just let it run on the main thread, it seems to be working fine and never crashing. Thus, could it be because I'm running this on a different thread that this is crashing?
I somehow find that weird because I don't believe I'm breaking any Core Data rules regarding multi-threading. In particular, I'm not passing any managed objects around, and whenever I need access to the MOC, I create a new MOC, i.e. I'm not relying on any MOC (or for that matter: anything) that has been created earlier on the main thread. Besides the little MOC stuff that occurs, occurs after the mapping model creation method, i.e. after the point at which the app crashes, so it can't possibly be a cause of the crashes under consideration here.
All I'm doing is taking two MOMs and asking for a mapping model between them. That can't be wrong even under threading, now can it?
Any ideas on what could be going on?
We have an IIS 7.5 web service on Windows Server 2008 that handles WCF requests from C# clients. This service is configured to have Maximum Worker Processes = 1, so it is not a web garden.
IIS is setup to recycle itself at the same time every day (3 AM). I am trying to debug gnarly connection issues, so I wanted to be sure the application pool was not recycling itself. I configured the pool to log an event when it recycles itself.
To my surprise, I see the following entries in the System event log:
Level: Information Date/Time: 3/23/2012 3:00:00 AM - Source: WAS - Event ID: 5076
A worker process with process id of '6636' serving application pool 'MyAppPool' has requested a recycle because it reached its scheduled recycle time.
Level: Information Date/Time: 3/23/2012 2:59:39 AM - Source: WAS - Event ID: 5076
A worker process with process id of '9364' serving application pool 'MyAppPool' has requested a recycle because it reached its scheduled recycle time.
IIS is correctly recycling the application pool at 3 AM. However, I do not understand why I would be getting two recycle events in the log within a few seconds of each other. The maximum number of processes is 1. Does IIS sometimes allocate multiple processes for an application pool that is specified as having 1 process?
-- edit --
I connected at about 4 PM today and only saw 1 w3wp.exe process. There are no other event log entries that would indicate a crash.
The popular PTS (Platform Technology Services) technical trainings for partners now include a workshop on Big Data.
First workshop will take place in Milan on July 10-12. (You can register by clicking the link below)
Oracle Big Data
Technical
Workshop
July
10-12,
2012: Cinisello Balsamo, Milan, Italy
For more info contact [email protected]
This Scenario: User copies large files to a local folder. I want to watch that folder and when a new file is dropped then go and copy it to Database, so later when coping is done I can actually use it in my application. ( A C# WinForms App). It would be awesome to also find a way to somehow get notified in the Application that hey copying the file to DB is finished and ready for use...
I am using C#.net, Windows... What solutions/architecture do you suggest for this?
For example having a windows service running all the time watching that folder, when something copied goes and write it to DB ... then how about getting notified? MSMQ is something I can use? don't know much about it yet.
Thanks.
Oracle's Master Data Management suite has seen remarkable development progress in the past year and a half. Leveraging out-of-the-box integration to applications provided by Application Integration Architecture, the cost, risk and time it takes to implement an MDM solution has been cut in half. Oracle Applications are now 'MDM Aware', Data Quality tools have reached state-of-the-art status, and new hubs are coming on line. In this AppsCast, Pascal Laik, VP MDM Products discusses this progress, what it means for Oracle customers, and where we are going from here.
I have installed MonoDevelop and after that Wine, but when I want to open an EXE file and then double click on it, it always runs with the Mono runtime, and if I want to run it with Wine, I should right click on it and choose Wine.
Well, in properties and in Open With there isn't Wine that choose it for the default application... I have this problem for all types of applications and files.
When the software you want to run is in the right click application (Open With),
I can't choose it in properties for the default. In fact, I can't find it in properties, because it is in open with option in right click...
What should I do to fix this problem?
I'm running Ubuntu 12.10 (Quantal Quetzal), 32 bit.
In my last question i asked how to best send a string from one view controller to another, both which were on a navigation stack:
http://stackoverflow.com/questions/2898860/pass-string-from-tableviewcontroller-to-viewcontroller-in-navigation-stack
However I just realised I can either pass the path to the file in the app's document's folder as the first (the table view) has already accessed the data in the file should I pass viewcontroller the data to the pushed VC?
This brief post is really intended as more of a reference than an article. The table below highlights two things, firstly where you can add you own custom logic via groovy code (end column), and secondly (middle column) when you might use each particular feature. Obviously this applies only where Application Composer exists, namely Fusion CRM and Oracle Sales Cloud, and is based on current (release 8) functionality.
Feature
Most Common Use Case
Groovy
Field Triggers
React to run-time data changes. Only fired when the field is changed and upon submit.
Y
Object Triggers
To extend the standard processing logic for an object, based on record creation, updates and deletes. There is a split between these firing events, with some related to UI/ADF actions and others originating in the database.
UI Trigger Points:
After Create - fires when a new object record is created. Commonly used to set default values for fields.
Before Modify - Fires when the end-user tries to modify a field value. Could be used for generic warnings or extra security logic.
Before Invalidate - Fires on the parent object when one of its child object records is created, updated, or deleted. For building in relationship logic.
Before Remove - Fires when an attempt is made to delete an object record. Can be used to create conditions that prevent deletes.
Database Trigger Points:
Before Insert in Database - Fires before a new object is inserted into the database. Can be used to ensure a dependent record exists or check for duplicates.
After Insert in Database - Fires after a new object is inserted into the database. Could be used to create a complementary record.
Before Update in Database -Fires before an existing object is modified in the database. Could be used to check dependent record values.
After Update in Database - Fires after an existing object is modified in the database. Could be used to update a complementary record.
Before Delete in Database - Fires before an existing object is deleted from the database. Could be used to check dependent record values.
After Delete in Database - Fires after an existing object is deleted from the database. Could be used to remove dependent records.
After Commit in Database - Fires after the change pending for the current object (insert, update, delete) is made permanent in the current transaction. Could be used when committed data that has passed all validation is required.
After Changes Posted to Database - Fires after all changes have been posted to the database, but before they are permanently committed. Could be used to make additional changes that will be saved as part of the current transaction.
Y
Field Validation
Displays a user entered error message based groovy logic validating the field value. The message is shown only when the validation logic returns false, and the logic is triggered only when tabbing out of the field on the user interface.
Y
Object Validation
Commonly used where validation is needed across multiple related fields on the object. Triggered on the submit UI action.
Y
Object Workflows
All Object Workflows are fired upon either record creation or update, along with the option of adding a custom groovy firing condition.
Y
Field Updates - change another field when a specified one changes.
Intended as an easy way to set different run-time values (e.g. pick
values for LOV's) plus the value field permits groovy logic entry.
Y
E-Mail Notification - sends an email notification to specified
users/roles. Templates support using run-time value tokens and rich
text.
N
Task Creation - for adding standard tasks for use in the worklist functionality.
N
Outbound Message - will create and send an XML payload of the related object SDO to a specified endpoint.
N
Business Process Flow - intended for approval using the seeded process, however can also trigger custom BPMN flows.
N
Global Functions
Utility functions that can be called from any groovy code in Application Composer (across applications).
Y
Object Functions
Utility functions that are local to the parent object. Usually triggered from within 'Buttons and Actions' definitions in Application Composer, although can be called from other code for that object (e.g. from a trigger).
Y
Add Custom Fields
When adding custom fields there are a few places you can include groovy logic.
Y
Default Value - to add logic within setting the default value when new records are entered.
Y
Conditionally Updateable - to add logic to set the field to read-only or not.
Y
Conditionally Required - to add logic to set the field to required or not.
Y
Formula Field - Used to provide a new aggregate field that is entirely based on groovy logic and other field values.
Y
Simplified UI Layouts - Advanced Expressions
Used for creating dynamic layouts for simplified UI pages where fields and regions show/hide based on run-time context values and logic. Also includes support for the depends-on feature as a trigger.
Y
Related References
This Blog: Application Composer Series
Extending Sales Guide: Using Groovy Scripts
Groovy Scripting Reference Guide
My client wants to standardize address information for existing and future addresses collected for their customers, particularly the street suffixes. The application used to enter and collect address information has the street suffix separated from the address field, but it is a textbox instead of a drop down list therefore things are not standardized. I know there are some options out there to standarize data, but they would like a less expensive alternative. Are there any functions in SQL Server that I can use to standardized data?
Choosing a long term data storage medium isn';t as easy as you may think. You might imagine that the data could be burnt to CD, locked in a cupboard and that it would last forever however unfortunatel... [Author: Chris Holgate - Computers and Internet - April 02, 2010]
States are passing more and more data security laws, the US Senate and the House have bills meandering through Congress, securing personal information and encrypting that data is no longer optional.
I have a data.frame:
df<-data.frame(a=c("x","x","y","y"),b=c(1,2,3,4))
> df
a b
1 x 1
2 x 2
3 y 3
4 y 4
What's the easiest way to print out each pair of values as a list of strings like this:
"x1", "x2", "y1", "y2"
I would like to invite you to a fascinating webinar on extending applications to HTML5 and Mobile that we are doing in collaboration with Jeffrey S. Hammond, Principal Analyst serving Application Development & Delivery Professionals at Forrester Research.The webinar is free and it will will introduce the substantial changes brought on by the move to Web Applications and Open Web architectures, and the challenges it places on application development shops. We’ll also introduce how we at Gizmox are helping client navigate this mobile shift and evolve existing Windows applications with a new set of Transposition tools called Instant CloudMove. We will discuss the alternatives in the market to evolve your existing applications and focus on our transposition tools that reduce migration risk, minimize costs, and accelerate your time to market.
So if you have locally installed Windows, VB6 or ASP applications that you are looking to enable as SaaS, offer over private or public Cloud platforms or allow end users with mobile accessibility then you shouldn't miss this webinar.
Extending Windows Applications to HTML5 and Mobile Has Never Been Easier
Tuesday, April 24, 2012 1:00 PM - 2:00 PM EST
Free registration:http://www.visualwebgui.com/Gizmox/Landing/tabid/674/articleType/ArticleView/articleId/987/Extending-Windows-Applications-to-HTML5-and-Mobile-Has-Never-Been-Easier.aspx
I have a great idea for smartphone application, And I want to develop an application suited for both android and iPhone. In addition I need to use spatial database for geo indexing that will be shared for both applications. I am new to this app world and I have some questions.
Is there away to develop for both machines? I know java but not objective c.
My guess is that I need to separate the database from the computing to support both applications.
What are the best cloud computing providers with spatial database support that can host the server?
Do I need 2 hosting servers or there is one server the can support the both of them?
which database provider can support geo indexing and support this intergraion,
I prefer providers with reasonable free quotas.
Thanks.