Search Results

Search found 282 results on 12 pages for 'aggregation'.

Page 4/12 | < Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >

• Temperature anomaly calculation of time series data

  - by neel

  I have a time series like following: Data <- structure(list(Year = c(1991L, 1991L, 1991L, 1991L, 1991L, 1991L, 1991L, 1991L, 1991L, 1991L, 1991L, 1991L, 1991L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L, 1992L), Month = c(8L, 9L, 9L, 9L, 10L, 10L, 10L, 11L, 11L, 11L, 12L, 12L, 12L, 1L, 1L, 1L, 2L, 2L, 2L, 3L, 3L, 3L, 4L, 4L, 4L, 5L, 5L, 5L, 6L, 6L, 6L, 7L, 7L, 7L, 8L, 8L, 8L, 9L, 9L, 9L, 10L, 10L, 10L, 11L, 11L, 11L, 12L, 12L, 12L), Day = c(30L, 10L, 20L, 30L, 10L, 20L, 30L, 10L, 20L, 30L, 10L, 20L, 30L, 10L, 20L, 30L, 10L, 20L, 28L, 10L, 20L, 30L, 10L, 20L, 30L, 10L, 20L, 30L, 10L, 20L, 30L, 10L, 20L, 30L, 10L, 20L, 30L, 10L, 20L, 30L, 10L, 20L, 30L, 10L, 20L, 30L, 10L, 20L, 30L), Hour = c(0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L), temperature = c(72.5, 64, 62.5, 64, 64, 53, 52, 52, 45.5, 49, 50, 50, 59, 63.5, 69.5, 61, 61, NaN, NaN, 39.5, 37, 45.5, 45, 39, 43.5, 52, 53, 56, 64, 66, 66.5, 73.5, 81, 85, 89.5, 87.5, 88.5, 83, 84.5, 74, 60.5, 59, 53, 60.5, 62.5, 64.5, 63, 62, 65.5)), .Names = c("Year", "Month", "Day", "Hour", "temperature"), class = "data.frame", row.names = c(NA, -49L)) and I have to calculate standardized anomaly. The steps to calculate the anomalies are following: Monthly premature departures from the long-term (1991-2007) average are obtained. Then standardized by dividing by the standard deviation of monthly temperature. The standardized monthly anomalies are then weighted by multiplying by the fraction of the average temperature for the given month. These weighted anomalies are then summed over 3 month time period. Can you please help me?

  Read the article

• How To Aggregate API Data?

  - by Mindblip

  Hi, I have a system that connects to 2 popular APIs. I need to aggregate the data from each into a unified result that can then be paginated. The scope of the project means that the system could end up supporting 10's of APIs. Each API imposes a max limit of 50 results per request. What is the best way of aggregating this data so that it is reliable i.e ordered, no duplicates etc I am using CakePHP framework on a LAMP environment, however, I think this question relates to all programming languages. My approach so far is to query the search API of each provider and then populate a MySQL table. From this the results are ordered, paginated etc. However, my concern is performance: API communication, parsing, inserting and then reading all in one execution. Am I missing something, does anyone have any other ideas? I'm sure this is a common problem with many alternative solutions. Any help would be greatly appreciated. Thanks, Paul

  Read the article

• Open Source Log Aggregators?

  - by Dean J

  I have sixteen servers using Log4J logs, accessible by ssh. I want to see the output of all logs on my desktop machine. Apache Chainsaw can presumably do this, but the documentation isn't getting me there. "Put all the jars into your ~/.chainsaw directory", got that. "Chainsaw will automatically use the functionality in those JARs"? Nope. Chainsaw isn't picking up log4j-chainsaw-vfs.jar, by the look of it, so sftp is out. Any suggestions other than Chainsaw?

  Read the article

• How do I use accepts_nested_attributes_for? I cannot use the .build method (!)

  - by Angela

  Editing my question for conciseness and to update what I've done: How do I model having multiple Addresses for a Company and assign a single Address to a Contact, and be able to assign them when creating or editing a Contact? Here is my model for Contacts: class Contact < ActiveRecord::Base attr_accessible :first_name, :last_name, :title, :phone, :fax, :email, :company, :date_entered, :campaign_id, :company_name, :address_id, :address_attributes belongs_to :company belongs_to :address accepts_nested_attributes_for :address end Here is my model for Address: class Address < ActiveRecord::Base attr_accessible :street1, :street2, :city, :state, :zip has_many :contacts end I would like, when creating an new contact, access all the Addresses that belong to the other Contacts that belong to the Company. So here is how I represent Company: class Company < ActiveRecord::Base attr_accessible :name, :phone, :addresses has_many :contacts has_many :addresses, :through => :contacts end Here is how I am trying to create a field in the View for _form for Contact so that, when someone creates a new Contact, they pass the address to the Address model and associate that address to the Contact: <% f.fields_for :address, @contact.address do |builder| %> <p> <%= builder.label :street1, "Street 1" %> </br> <%= builder.text_field :street1 %> <p> <% end %> When I try to Edit, the field for Street 1 is blank. And I don't know how to display the value from show.html.erb. At the bottom is my error console -- can't seem to create values in the address table: My Contacts controller is as follows: def new @contact = Contact.new @contact.address.build # I GET AN ERROR HERE: says NIL CLASS @contact.date_entered = Date.today @campaigns = Campaign.find(:all, :order => "name") if params[:campaign_id].blank? else @campaign = Campaign.find(params[:campaign_id]) @contact.campaign_id = @campaign.id end if params[:company_id].blank? else @company = Company.find(params[:company_id]) @contact.company_name = @company.name end end def create @contact = Contact.new(params[:contact]) if @contact.save flash[:notice] = "Successfully created contact." redirect_to @contact else render :action => 'new' end end def edit @contact = Contact.find(params[:id]) @campaigns = Campaign.find(:all, :order => "name") end Here is a snippet of my error console: I am POSTING the attribute, but it is not CREATING in the Address table.... Processing ContactsController#create (for 127.0.0.1 at 2010-05-12 21:16:17) [POST] Parameters: {"commit"="Submit", "authenticity_token"="d8/gx0zy0Vgg6ghfcbAYL0YtGjYIUC2b1aG+dDKjuSs=", "contact"={"company_name"="Allyforce", "title"="", "campaign_id"="2", "address_attributes"={"street1"="abc"}, "fax"="", "phone"="", "last_name"="", "date_entered"="2010-05-12", "email"="", "first_name"="abc"}} Company Load (0.0ms)[0m [0mSELECT * FROM "companies" WHERE ("companies"."name" = 'Allyforce') LIMIT 1[0m Address Create (16.0ms)[0m [0;1mINSERT INTO "addresses" ("city", "zip", "created_at", "street1", "updated_at", "street2", "state") VALUES(NULL, NULL, '2010-05-13 04:16:18', NULL, '2010-05-13 04:16:18', NULL, NULL)[0m Contact Create (0.0ms)[0m [0mINSERT INTO "contacts" ("company", "created_at", "title", "updated_at", "campaign_id", "address_id", "last_name", "phone", "fax", "company_id", "date_entered", "first_name", "email") VALUES(NULL, '2010-05-13 04:16:18', '', '2010-05-13 04:16:18', 2, 2, '', '', '', 5, '2010-05-12', 'abc', '')[0m

  Read the article

• How do I use accepts_nested_attributes_for?

  - by Angela

  Editing my question for conciseness and to update what I've done: How do I model having multiple Addresses for a Company and assign a single Address to a Contact, and be able to assign them when creating or editing a Contact? I want to use nested attributes to be able to add an address at the time of creating a new contact. That address exists as its own model because I may want the option to drop-down from existing addresses rather than entering from scratch. I can't seem to get it to work. I get a undefined method `build' for nil:NilClass error Here is my model for Contacts: class Contact < ActiveRecord::Base attr_accessible :first_name, :last_name, :title, :phone, :fax, :email, :company, :date_entered, :campaign_id, :company_name, :address_id, :address_attributes belongs_to :company belongs_to :address accepts_nested_attributes_for :address end Here is my model for Address: class Address < ActiveRecord::Base attr_accessible :street1, :street2, :city, :state, :zip has_many :contacts end I would like, when creating an new contact, access all the Addresses that belong to the other Contacts that belong to the Company. So here is how I represent Company: class Company < ActiveRecord::Base attr_accessible :name, :phone, :addresses has_many :contacts has_many :addresses, :through => :contacts end Here is how I am trying to create a field in the View for _form for Contact so that, when someone creates a new Contact, they pass the address to the Address model and associate that address to the Contact: <% f.fields_for :address, @contact.address do |builder| %> <p> <%= builder.label :street1, "Street 1" %> </br> <%= builder.text_field :street1 %> <p> <% end %> When I try to Edit, the field for Street 1 is blank. And I don't know how to display the value from show.html.erb. At the bottom is my error console -- can't seem to create values in the address table: My Contacts controller is as follows: def new @contact = Contact.new @contact.address.build # Iundefined method `build' for nil:NilClass @contact.date_entered = Date.today @campaigns = Campaign.find(:all, :order => "name") if params[:campaign_id].blank? else @campaign = Campaign.find(params[:campaign_id]) @contact.campaign_id = @campaign.id end if params[:company_id].blank? else @company = Company.find(params[:company_id]) @contact.company_name = @company.name end end def create @contact = Contact.new(params[:contact]) if @contact.save flash[:notice] = "Successfully created contact." redirect_to @contact else render :action => 'new' end end def edit @contact = Contact.find(params[:id]) @campaigns = Campaign.find(:all, :order => "name") end Here is a snippet of my error console: I am POSTING the attribute, but it is not CREATING in the Address table.... Processing ContactsController#create (for 127.0.0.1 at 2010-05-12 21:16:17) [POST] Parameters: {"commit"="Submit", "authenticity_token"="d8/gx0zy0Vgg6ghfcbAYL0YtGjYIUC2b1aG+dDKjuSs=", "contact"={"company_name"="Allyforce", "title"="", "campaign_id"="2", "address_attributes"={"street1"="abc"}, "fax"="", "phone"="", "last_name"="", "date_entered"="2010-05-12", "email"="", "first_name"="abc"}} Company Load (0.0ms)[0m [0mSELECT * FROM "companies" WHERE ("companies"."name" = 'Allyforce') LIMIT 1[0m Address Create (16.0ms)[0m [0;1mINSERT INTO "addresses" ("city", "zip", "created_at", "street1", "updated_at", "street2", "state") VALUES(NULL, NULL, '2010-05-13 04:16:18', NULL, '2010-05-13 04:16:18', NULL, NULL)[0m Contact Create (0.0ms)[0m [0mINSERT INTO "contacts" ("company", "created_at", "title", "updated_at", "campaign_id", "address_id", "last_name", "phone", "fax", "company_id", "date_entered", "first_name", "email") VALUES(NULL, '2010-05-13 04:16:18', '', '2010-05-13 04:16:18', 2, 2, '', '', '', 5, '2010-05-12', 'abc', '')[0m

  Read the article

• SQL one table aggrigation

  - by Lostdrifter

  Ok, for the last few days I have been attempting to find a method to pull a very important set of information form a table that contains what I call daily counts. I have a table that is setup as follows. person|company|prod1|prod2|prod3|gen_date Each company has more then one person, and each person can have different combination of products that they have purchased. What I have been trying to figure out is a SQL statement that will list the number of people that have bought a particular product per company. So an output similar to this: Comp ABC | 13 Prod1 | 3 Prod2 | 5 Prod 3 Comp DEF | 2 Prod1 | 15 Prod2 | 0 Prod 3 Comp HIJ | 0 Prod1 | 0 Prod2 | 7 Prod 3 Currently if a person did not select a product the value being stored is NULL. Best I have right now is 3 different statements that can produce this information if run on there own. SELECT Count(person) as puchases, company FROM Sales WHERE prod1 = '1' and gendate = '3/24/2010' Group BY company

  Read the article

• Calculated group-by fields in MongoDB

  - by Navin Viswanath

  For this example from the MongoDB documentation, how do I write the query using MongoTemplate? db.sales.aggregate( [ { $group : { _id : { month: { $month: "$date" }, day: { $dayOfMonth: "$date" }, year: { $year: "$date" } }, totalPrice: { $sum: { $multiply: [ "$price", "$quantity" ] } }, averageQuantity: { $avg: "$quantity" }, count: { $sum: 1 } } } ] ) Or in general, how do I group by a calculated field?

  Read the article

• MongoDb - $match filter not working in subdocument

  - by Ranjith

  This is Collection Structure [{ "_id" : "....", "name" : "aaaa", "level_max_leaves" : [ { level : "ObjectIdString 1", max_leaves : 4, } ] }, { "_id" : "....", "name" : "bbbb", "level_max_leaves" : [ { level : "ObjectIdString 2", max_leaves : 2, } ] }] I need to find the subdocument value of level_max_leaves.level filter when its matching with given input value. And this how I tried, For example, var empLevelId = 'ObjectIdString 1' ; MyModel.aggregate( {$unwind: "$level_max_leaves"}, {$match: {"$level_max_leaves.level": empLevelId } }, {$group: { "_id": "$level_max_leaves.level", "total": { "$sum": "$level_max_leaves.max_leaves" }}}, function (err, res) { console.log(res); }); But here the $match filter is not working. I can't find out exact results of ObjectIdString 1 If I filter with name field, its working fine. like this, {$match: {"$name": "aaaa" } }, But in subdocument level its returns 0. {$match: {"$level_max_leaves.level": "ObjectIdString 1"} }, My expected result was, { "_id" : "ObjectIdString 1", "total" : 4, }

  Read the article

• Django Include Aggregate Sums of Zero

  - by tomas

  I'm working on a Django timesheet application and am having trouble figuring out how to include aggregate sums that equal zero. If I do something like: entries = TimeEntry.objects.all().values("user__username").annotate(Sum("hours")) I get all users that have time entries and their sums. [{'username': u'bob' 'hours__sum':49}, {'username': u'jane' 'hours__sum':10}] When I filter that by a given day: filtered_entries = entries.filter(date="2010-05-17") Anyone who didn't enter time for that day is excluded. Is there a way to include those users who's sums are 0? Thanks

  Read the article

• How can I remove a duplicate object from a MongoDB array?

  - by andrewrk

  My data looks like this: foo_list: [ { id: '98aa4987-d812-4aba-ac20-92d1079f87b2', name: 'Foo 1', slug: 'foo-1' }, { id: '98aa4987-d812-4aba-ac20-92d1079f87b2', name: 'Foo 1', slug: 'foo-1' } { id: '157569ec-abab-4bfb-b732-55e9c8f4a57d', name: 'Foo 3', slug: 'foo-3' } ] Where foo_list is a field in a model called Bar. Notice that the first and second objects in the array are complete duplicates. Aside from the obvious solution of switching to PostgresSQL, what MongoDB query can I run to remove duplicate entries from foo_list? Similar answers that do not quite cut it: http://stackoverflow.com/a/16907596/432 http://stackoverflow.com/a/18804460/432 These questions answer the question if the array had bare strings in it. However in my situation the array is filled with objects. I hope it is clear that I am not interested in a query; I want the duplicates to be gone from the database forever.

  Read the article

• How can I add two models in one form, where one model is a has_many :through?

  - by Angela

  How do I model having multiple Addresses for a Company and assign a single Address to a Contact? Contacts belong_to a Company. A Company has_many Contacts. A Company also has_many Addresses. And each Contact has_one Address. I want to be able, whenever I create a New Contact, to access all the addresses in all Contacts that belong to the Company. Here is are my Models: class Company < ActiveRecord::Base attr_accessible :name, :phone, :addresses has_many :contacts has_many :addresses, :through => :contacts end class Contact < ActiveRecord::Base attr_accessible :first_name, :last_name, :title, :phone, :fax, :email, :company, :date_entered, :campaign_id, :company_name, :address belongs_to :company has_one :address accepts_nested_attributes_for :address end class Address < ActiveRecord::Base attr_accessible :street1 has_many :contacts end How do I create the View in the _form for Contacts so that I can 1) Add an Address when creating a Contact; 2) Display the options of the Address. Here is how I am doing step 1, which is just to add a new address for a new contact: <% f.fields_for :addresses do |builder| %> <p> <%= builder.label :street1, "Street 1" %> </br> <%= builder.text_field :street1 %> <p> Right now, what I have doesn't work. The console says I cannot mass-assign addresses when I hit "submit" on this New contact form.

  Read the article

• Saving associated domain classes in Grails

  - by Cesar

  I'm struggling to get association right on Grails. Let's say I have two domain classes: class Engine { String name int numberOfCylinders = 4 static constraints = { name(blank:false, nullable:false) numberOfCylinders(range:4..8) } } class Car { int year String brand Engine engine = new Engine(name:"Default Engine") static constraints = { engine(nullable:false) brand(blank:false, nullable:false) year(nullable:false) } } The idea is that users can create cars without creating an engine first, and those cars get a default engine. In the CarController I have: def save = { def car = new Car(params) if(!car.hasErrors() && car.save()){ flash.message = "Car saved" redirect(action:index) }else{ render(view:'create', model:[car:car]) } } When trying to save, I get a null value exception on the Car.engine field, so obviously the default engine is not created and saved. I tried to manually create the engine: def save = { def car = new Car(params) car.engine = new Engine(name: "Default Engine") if(!car.hasErrors() && car.save()){ flash.message = "Car saved" redirect(action:index) }else{ render(view:'create', model:[car:car]) } } Didn't work either. Is Grails not able to save associated classes? How could I implement such feature?

  Read the article

• UML aggregation. Is necessary to put an atributte.

  - by VansFannel

  Hello. I'm making an UML class diagram and I don't know how to do this: I have a content that has a file. Both, content and file, are represented as classes. I've put an aggregation between content and file. But I don't know if is necessary to put an atribute inside content to represent file. Is necessary? Thank you.

  Read the article

• What is a common name for inheritance, composition, aggregation, delegation?

  - by Eye of Hell

  Hello. After program is separated into small object, these objects must be connected with each over. Where are different types of connection. Inheritance, composition, aggregation, delegation. These types has many kinds and patterns like loose coupling, tight coupling, inversion of control, delegation via interfaces etc. What is a correct common name for mentioned types of connections? I can suggest that they all are called 'coupling', but i can't find any good classification in google, so maybe i'm trying to use a wrong term? Maybe anyone knows a solid, trusted classification that i can user for terminology?

  Read the article

• Show composition/aggregation/association relations between objects in Visual Paradigm UML diagrams?

  - by ajsie

  I have Netbeans installed with Visual Paradigm plugin. I have converted my php code into UML diagrams (modeling - instant reverse). I can see relations (drawn lines) between superclass and subclasses. However, i cannot see relations between objects inside objects (composition/aggregation/association)? The code looks like: class Thread { private $tag = ''; public function __construct($tagObject) { $this->tag = $tagObject; } } I know its possible using Java cause i've read about it. Im using PHP, is this still possible?

  Read the article

• data handling with javascript

  - by Vincent Warmerdam

  Python has a very neat package called pandas which allows for quick data transformation; tables, aggregation, that sort of thing. A lot of these types of functionality can also be found in the python itertools module. The plyR package in R is also very similar. Usually one woud use this functionality to produce a table which is later visualized with a plot. I am personally very fond of d3, and I would like to allow the user to first indicate what type of data aggregation he wants on the dataset before it is visualized. The visualisation in question involves making a heatmap where the user gets to select the size of the bins of the heatmap beforehand (I want d3 to project this through leaflet). I want to visually select the ideal size of the bins for the heatmap. The way I work now is that I take the dataset, aggregate it with python and then manually load it in d3. This is a process that takes a lot of human effort and I was wondering if the data aggregation can be done through the javascript of the browser. I couldn't find a package for javascript specifically built for data, suggesting (to me) that this is a bad idea and that one should not use javascript for the data handling. Is there a good module/package for javascript to handle data aggregation? Is it a good/bad idea to do the data aggregation in javascript (performance wise)?

  Read the article

• How to configure multiple VLAN's on a lacp link aggregation on openindiana(oi_151a7)?

  - by reco

  i have 4 links aggregated. $ dladm show-link LINK CLASS MTU STATE BRIDGE OVER igb0 phys 1500 up -- -- igb1 phys 1500 up -- -- igb2 phys 1500 up -- -- igb3 phys 1500 up -- -- aggr0 aggr 1500 up -- igb0 igb1 igb2 igb3 i managed to create one VLAN on the aggr0 link: $ dladm show-vlan LINK VID OVER FLAGS vlan1 9 aggr0 ----- if i try to add more i get the following error: $ dladm create-vlan -v 3 -l aggr0 vlan2 dladm: create operation failed: invalid argument

  Read the article

• Suggestion for live video stream aggregation/switching/forwarding/management software?

  - by deceze

  I'm looking for a software or system that can receive video streams from a number of cameras via a network (RTMP or similar protocol), present a visual overview of all video streams and allow me to forward/send a selected stream to another service (e.g. to a Flash Media Server, or anywhere via RTMP). Basically the digital internet equivalent of a TV studio control panel, which allows a director to put together a live show. Is there any such software at an affordable price? A GUI-less server which can be scripted to switch streams would be good too. I'm not even quite sure what kind of product category this falls into or what search terms to plug into Google. Most results I have come up with have little more than an executive summary description which doesn't tell me anything. Any suggestion welcome.

  Read the article

• ODI 12c - Aggregating Data

  - by David Allan

  This posting will look at the aggregation component that was introduced in ODI 12c. For many ETL tool users this shouldn't be a big surprise, its a little different than ODI 11g but for good reason. You can use this component for composing data with relational like operations such as sum, average and so forth. Also, Oracle SQL supports special functions called Analytic SQL functions, you can use a specially configured aggregation component or the expression component for these now in ODI 12c. In database systems an aggregate transformation is a transformation where the values of multiple rows are grouped together as input on certain criteria to form a single value of more significant meaning - that's exactly the purpose of the aggregate component. In the image below you can see the aggregate component in action within a mapping, for how this and a few other examples are built look at the ODI 12c Aggregation Viewlet here - the viewlet illustrates a simple aggregation being built and then some Oracle analytic SQL such as AVG(EMP.SAL) OVER (PARTITION BY EMP.DEPTNO) built using both the aggregate component and the expression component. In 11g you used to just write the aggregate expression directly on the target, this made life easy for some cases, but it wan't a very obvious gesture plus had other drawbacks with ordering of transformations (agg before join/lookup. after set and so forth) and supporting analytic SQL for example - there are a lot of postings from creative folks working around this in 11g - anything from customizing KMs, to bypassing aggregation analysis in the ODI code generator. The aggregate component has a few interesting aspects. 1. Firstly and foremost it defines the attributes projected from it - ODI automatically will perform the grouping all you do is define the aggregation expressions for those columns aggregated. In 12c you can control this automatic grouping behavior so that you get the code you desire, so you can indicate that an attribute should not be included in the group by, that's what I did in the analytic SQL example using the aggregate component. 2. The component has a few other properties of interest; it has a HAVING clause and a manual group by clause. The HAVING clause includes a predicate used to filter rows resulting from the GROUP BY clause. Because it acts on the results of the GROUP BY clause, aggregation functions can be used in the HAVING clause predicate, in 11g the filter was overloaded and used for both having clause and filter clause, this is no longer the case. If a filter is after an aggregate, it is after the aggregate (not sometimes after, sometimes having).  3. The manual group by clause let's you use special database grouping grammar if you need to. For example Oracle has a wealth of highly specialized grouping capabilities for data warehousing such as the CUBE function. If you want to use specialized functions like that you can manually define the code here. The example below shows the use of a manual group from an example in the Oracle database data warehousing guide where the SUM aggregate function is used along with the CUBE function in the group by clause. The SQL I am trying to generate looks like the following from the data warehousing guide; SELECT channel_desc, calendar_month_desc, countries.country_iso_code,       TO_CHAR(SUM(amount_sold), '9,999,999,999') SALES$ FROM sales, customers, times, channels, countries WHERE sales.time_id=times.time_id AND sales.cust_id=customers.cust_id AND   sales.channel_id= channels.channel_id  AND customers.country_id = countries.country_id  AND channels.channel_desc IN   ('Direct Sales', 'Internet') AND times.calendar_month_desc IN   ('2000-09', '2000-10') AND countries.country_iso_code IN ('GB', 'US') GROUP BY CUBE(channel_desc, calendar_month_desc, countries.country_iso_code); I can capture the source datastores, the filters and joins using ODI's dataset (or as a traditional flow) which enables us to incrementally design the mapping and the aggregate component for the sum and group by as follows; In the above mapping you can see the joins and filters declared in ODI's dataset, allowing you to capture the relationships of the datastores required in an entity-relationship style just like ODI 11g. The mix of ODI's declarative design and the common flow design provides for a familiar design experience. The example below illustrates flow design (basic arbitrary ordering) - a table load where only the employees who have maximum commission are loaded into a target. The maximum commission is retrieved from the bonus datastore and there is a look using employees as the driving table and only those with maximum commission projected. Hopefully this has given you a taster for some of the new capabilities provided by the aggregate component in ODI 12c. In summary, the actions should be much more consistent in behavior and more easily discoverable for users, the use of the components in a flow graph also supports arbitrary designs and the tool (rather than the interface designer) takes care of the realization using ODI's knowledge modules. Interested to know if a deep dive into each component is interesting for folks. Any thoughts? 

  Read the article

• Understanding UML composition better

  - by Prog

  The technical difference between Composition and Aggregation in UML (and sometimes in programming too) is that with Composition, the lifetime of the objects composing the composite (e.g. an engine and a steering wheel in a car) is dependent on the composite object. While with Aggregation, the lifetime of the objects making up the composite is independent of the composite. However I'm not sure about something related to composition in UML. Say ClassA is composed of an object of ClassB: class ClassA{ ClassB bInstance; public ClassA(){ bInstance = new ClassB(); } } This is an example of composition, because bInstance is dependent on the lifetime of it's enclosing object. However, regarding UML notation - I'm not sure if I would notate the relationship between ClassA and ClassB with a filled diamond (composition) or a white diamond (aggregation). This is because while the lifetime of some ClassB instances is dependent of ClassA instances - there could be ClassB instances anywhere else in the program - not only within ClassA instances. The question is: if ClassA objects are composed of ClassB objects - but other ClassB objects are free to be used anywhere else in the program: Should the relationship between ClassA and ClassB be notated as aggregation or as composition?

  Read the article

• T-SQL Tuesday: Aggregations in SSIS

  - by andyleonard

  Introduction Jes Borland ( Blog | @grrl_geek ) is hosting this month's T-SQL Tuesday - started by SQLBlog's own Adam Machanic ( Blog | @AdamMachanic ) - and it is about aggregation. I thought I'd show a couple ways to do aggregation using SSIS. The Aggregate Transformation in SSIS The Aggregate transform in SSIS is fast . I built an SSIS package (AggregateScripts.dtsx) with two Data Flow Tasks (Using the Aggregate Transform and Using a Script Component). Using the Aggregate Transform looks like this:...(read more)

  Read the article

• How to aggregate data in Windows Performance Monitor?

  - by Jader Dias

  I have seem performance counters like this: PhysicalDisk(_Total)\Avg. Disk Read Queue Length Which is an aggregation (average, sum, etc) of data over a period. How to create a custom counter which is an aggregation of existent data of another counter?

  Read the article

• SEO effect of “You are leaving this site” page for outbound links?

  - by Timo Huovinen

  The problem I am working on an aggregation website that collects reviews about specific products from various websites. The site has many thousands of outbound links (with "nofollow" attributes) to the content source websites where the reviews were collected from. The site has far more outbound links than inbound links and I have read that this is bad for SEO. The question Would adding an intermediate «You are leaving this site» disclaimer/warning page like this hurt search engine rankings? And can you provide any links about this topic? p.s. The exit page would be a POST form instead of a script, that notifies the user that he/she is leaving this site and provides a button to continue to the other website. p.p.s This kind of idea is implemented on many forums, aggregation websites with the purpose of warning the user that he/she is leaving this site and to block search engine bots from following those links because search bots do not submit forms.

  Read the article

• how to convert sql union to linq

  - by lowlyintern

  I have the following Transact SQL query using a union. I need some pointers as to how this would look in LINQ i.e some examples wouldbe nice or if anyone can recommend a good tutorial on UNIONS in linq. select top 10 Barcode, sum(ItemDiscountUnion.AmountTaken) from (SELECT d.Barcode,SUM(AmountTaken) AmountTaken FROM [Aggregation].[dbo].[DiscountPromotion] d GROUP BY d.Barcode UNION ALL SELECT i.Barcode,SUM(AmountTaken) AmountTaken FROM [Aggregation].[dbo].ItemSaleTransaction i group by i.Barcode) ItemDiscountUnion group by Barcode

  Read the article

• The blocking nature of aggregates

  - by Rob Farley

  I wrote a post recently about how query tuning isn’t just about how quickly the query runs – that if you have something (such as SSIS) that is consuming your data (and probably introducing a bottleneck), then it might be more important to have a query which focuses on getting the first bit of data out. You can read that post here.  In particular, we looked at two operators that could be used to ensure that a query returns only Distinct rows. and The Sort operator pulls in all the data, sorts it (discarding duplicates), and then pushes out the remaining rows. The Hash Match operator performs a Hashing function on each row as it comes in, and then looks to see if it’s created a Hash it’s seen before. If not, it pushes the row out. The Sort method is quicker, but has to wait until it’s gathered all the data before it can do the sort, and therefore blocks the data flow. But that was my last post. This one’s a bit different. This post is going to look at how Aggregate functions work, which ties nicely into this month’s T-SQL Tuesday. I’ve frequently explained about the fact that DISTINCT and GROUP BY are essentially the same function, although DISTINCT is the poorer cousin because you have less control over it, and you can’t apply aggregate functions. Just like the operators used for Distinct, there are different flavours of Aggregate operators – coming in blocking and non-blocking varieties. The example I like to use to explain this is a pile of playing cards. If I’m handed a pile of cards and asked to count how many cards there are in each suit, it’s going to help if the cards are already ordered. Suppose I’m playing a game of Bridge, I can easily glance at my hand and count how many there are in each suit, because I keep the pile of cards in order. Moving from left to right, I could tell you I have four Hearts in my hand, even before I’ve got to the end. By telling you that I have four Hearts as soon as I know, I demonstrate the principle of a non-blocking operation. This is known as a Stream Aggregate operation. It requires input which is sorted by whichever columns the grouping is on, and it will release a row as soon as the group changes – when I encounter a Spade, I know I don’t have any more Hearts in my hand. Alternatively, if the pile of cards are not sorted, I won’t know how many Hearts I have until I’ve looked through all the cards. In fact, to count them, I basically need to put them into little piles, and when I’ve finished making all those piles, I can count how many there are in each. Because I don’t know any of the final numbers until I’ve seen all the cards, this is blocking. This performs the aggregate function using a Hash Match. Observant readers will remember this from my Distinct example. You might remember that my earlier Hash Match operation – used for Distinct Flow – wasn’t blocking. But this one is. They’re essentially doing a similar operation, applying a Hash function to some data and seeing if the set of values have been seen before, but before, it needs more information than the mere existence of a new set of values, it needs to consider how many of them there are. A lot is dependent here on whether the data coming out of the source is sorted or not, and this is largely determined by the indexes that are being used. If you look in the Properties of an Index Scan, you’ll be able to see whether the order of the data is required by the plan. A property called Ordered will demonstrate this. In this particular example, the second plan is significantly faster, but is dependent on having ordered data. In fact, if I force a Stream Aggregate on unordered data (which I’m doing by telling it to use a different index), a Sort operation is needed, which makes my plan a lot slower. This is all very straight-forward stuff, and information that most people are fully aware of. I’m sure you’ve all read my good friend Paul White (@sql_kiwi)’s post on how the Query Optimizer chooses which type of aggregate function to apply. But let’s take a look at SQL Server Integration Services. SSIS gives us a Aggregate transformation for use in Data Flow Tasks, but it’s described as Blocking. The definitive article on Performance Tuning SSIS uses Sort and Aggregate as examples of Blocking Transformations. I’ve just shown you that Aggregate operations used by the Query Optimizer are not always blocking, but that the SSIS Aggregate component is an example of a blocking transformation. But is it always the case? After all, there are plenty of SSIS Performance Tuning talks out there that describe the value of sorted data in Data Flow Tasks, describing the IsSorted property that can be set through the Advanced Editor of your Source component. And so I set about testing the Aggregate transformation in SSIS, to prove for sure whether providing Sorted data would let the Aggregate transform behave like a Stream Aggregate. (Of course, I knew the answer already, but it helps to be able to demonstrate these things). A query that will produce a million rows in order was in order. Let me rephrase. I used a query which produced the numbers from 1 to 1000000, in a single field, ordered. The IsSorted flag was set on the source output, with the only column as SortKey 1. Performing an Aggregate function over this (counting the number of rows per distinct number) should produce an additional column with 1 in it. If this were being done in T-SQL, the ordered data would allow a Stream Aggregate to be used. In fact, if the Query Optimizer saw that the field had a Unique Index on it, it would be able to skip the Aggregate function completely, and just insert the value 1. This is a shortcut I wouldn’t be expecting from SSIS, but certainly the Stream behaviour would be nice. Unfortunately, it’s not the case. As you can see from the screenshots above, the data is pouring into the Aggregate function, and not being released until all million rows have been seen. It’s not doing a Stream Aggregate at all. This is expected behaviour. (I put that in bold, because I want you to realise this.) An SSIS transformation is a piece of code that runs. It’s a physical operation. When you write T-SQL and ask for an aggregation to be done, it’s a logical operation. The physical operation is either a Stream Aggregate or a Hash Match. In SSIS, you’re telling the system that you want a generic Aggregation, that will have to work with whatever data is passed in. I’m not saying that it wouldn’t be possible to make a sometimes-blocking aggregation component in SSIS. A Custom Component could be created which could detect whether the SortKeys columns of the input matched the Grouping columns of the Aggregation, and either call the blocking code or the non-blocking code as appropriate. One day I’ll make one of those, and publish it on my blog. I’ve done it before with a Script Component, but as Script components are single-use, I was able to handle the data knowing everything about my data flow already. As per my previous post – there are a lot of aspects in which tuning SSIS and tuning execution plans use similar concepts. In both situations, it really helps to have a feel for what’s going on behind the scenes. Considering whether an operation is blocking or not is extremely relevant to performance, and that it’s not always obvious from the surface. In a future post, I’ll show the impact of blocking v non-blocking and synchronous v asynchronous components in SSIS, using some of LobsterPot’s Script Components and Custom Components as examples. When I get that sorted, I’ll make a Stream Aggregate component available for download.

  Read the article

< Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >