Search Results

Search found 91480 results on 3660 pages for 'large data in sharepoint list'.

Page 18/3660 | < Previous Page | 14 15 16 17 18 19 20 21 22 23 24 25  | Next Page >

• [C++] STL list - how to find a list element by its object fields

  - by Dominic Bou-Samra

  I have a list: list<Unit *> UnitCollection; containing Unit objects, which has an accessor like: bool Unit::isUnit(string uCode) { if(this->unitCode == uCode) return true; else return false; } How do I search my UnitCollection list by uCode and return the corresponding element (preferably it's index). I have looked at the find() method, but i'm not sure you can pass a boolean method in instead of a searched item parameter if that makes sense.

  Read the article

• List filtering: list comprehension vs. lambda + filter

  - by Agos

  I happened to find myself having a basic filtering need: I have a list and I have to filter it by an attribute of the items. My code looked like this: list = [i for i in list if i.attribute == value] But then i thought, wouldn't it be better to write it like this? filter(lambda x: x.attribute == value, list) It's more readable, and if needed for performance the lambda could be taken out to gain something. Question is: are there any caveats in using the second way? Any performance difference? Am I missing the Pythonic Way™ entirely and should do it in yet another way (such as using itemgetter instead of the lambda)? Thanks in advance

  Read the article

• Would this predicate work ? this came on my quiz to find the position of element X in a data structure called list

  - by M.K

  example: position(1,list(1,list(2,nil)),Z). Z = 1. position(3,list(1,list(2,list(3,nil)),Z). Z = 3. where Z is the position of element X in a data structure of a list in the above format Here was my solution: position(X,list(nil),0). %empty list position(X,list(X,T),1). %list with X as head or first element position(X,list(H,T),Z):- position(X,list(T,nil),Z1), %X is in tail of list (H,T) Z is Z1 + 1.

  Read the article

• List.clear() followed by List.add() not working.

  - by Vincent

  I have the following C# Class/Function: class Hand { private List<Card> myCards = new List<Card>(); public void sortBySuitValue() { IEnumerable<Card> query = from s in myCards orderby (int)s.suit, (int)s.value select s; myCards = new List<Card>(); myCards.AddRange(query); } } On a card Game. This works fine, however, I had trouble at first, instead of using myCards = new List(); to 'reset' myCards, I would use myCards.clear(), however, once I called the clear function, I would not be able to call myCards.add() or myCards.addRange(). The count would stay at zero. Is my current approach good? Is using LINQ to sort my cards good/bad?

  Read the article

• Backup a hosted Sharepoint

  - by David Mackintosh

  One of my customers has outsourced their Sharepoint and Exchange services to a hosted services provider. I believe it is a Sharepoint 2007 service. It is a shared hosting solution, so we do not have any kind of access to the server itself; we only have user-level and sharepoint-administrator-level access to the Sharepoint application. They have come to the point where they would like to have a copy of everything that is on the Sharepoint server. I have downloaded the Office Sharepoint Designer 2007, and it features three (!) ways to backup a Sharepoint server, none (!) of which work for me: File-Export-Personal Web Package: When selecting everything, it calculates a negative size. Barfs with No "content-type" in CGI environment error. File-Export-Sharepoint Template: barfs with a A World Wide Web browser, such as Windows Internet Explorer, is required to use this feature error. Site-Administration-Backup Web Site: wants to create the backup .cmp file on the sharepoint server itself. I don't have access to any servers on the same network so I can't redirect it to any form of the suggested \\server\place. Barfs with a The Web application at $URL could not be found. [...] error. Possibly moot because Google tells me that bad things happen using OSD to back up sites larger than 24MB (which this site is most definitely). So I called the helpdesk of the outsource provider, and got told that they recommend using OSD, but no they don't actually provide any application support for OSD (not that I blame them for that), but they could do a stsadm.exe backup and provide us with that, and OSD should be able to read the resulting cmp file. Then for authorization reasons they had my customer call them directly (since I can't authorize such an operation), and they told him that he didn't want a stsadm.exe backup, he wanted to get into an 'explorer view' and deal with things that way (they were vague). Google hasn't been much help in figuring out what an 'explorer view' is, let alone how I bring one up. The end goal of this operation is to have a backup of the site as it exists (hopefully today, but shortly anyways) in such a format that we don't need another sharepoint server to restore it to. Ie we'd like to be able to pick individual content directly out of this backup. We are not excessively concerned with things like formatting. We just want the documents. This is a fairly complex site with multiple subsites and multiple folders per subsite, so sitting there and manually downloading each file isn't really going to happen if there is a better easier way. So, my questions: Is the stsadm.exe backup what I want? If not, what do I want? If I manage to convince them that I do want the stsadm.exe backup, can I pick files out of the resulting backup file with OSD? If OSD isn't going to let me extract individual files, is there a tool I can use that can?

  Read the article

• How to use SharePoint modal dialog box to display Custom Page Part3

  - by ybbest

  In the second part of the series, I showed you how to display and close a custom page in a SharePoint modal dialog using JavaScript and display a message after the modal dialog is closed. In this post, I’d like to show you how to use SPLongOperation with the Modal dialog box. You can download the source code here. 1. Firstly, modify the element file as follow   2. In your code behind, you can implement a close dialog function as below. This will close your modal dialog box once the button is clicked and display a status bar. Note that you need to use window.frameElement.commonModalDialogClose instead of window.frameElement.commonModalDialogClose References: How to: Display a Page as a Modal Dialog Box

  Read the article

• SharePoint Saturday Charlotte 2010 Recap, Slides and Photos

  - by Brian Jackett

  This past weekend I attended SharePoint Saturday Charlotte (SPSCLT) in Charlotte, North Carolina.  For those unfamiliar, SharePoint Saturday is a community driven event where various speakers gather to present at a FREE conference on all topics related to SharePoint.  This made my fourth SharePoint Saturday attended and third I’ve spoken at.  The event was very well organized, attended, and a pleasure to be a part of along with many other great speakers.     At SharePoint Saturday Charlotte I had the opportunity to give two presentations.  First was “The Power of PowerShell + SharePoint 2007” and second was a new one “Managing SharePoint 2010 Farms with PowerShell.”  I want to thank everyone who attended either of my sessions and for all of the feedback given.  Below you will find links to my slides, demo scripts, and pictures taken throughout the event.  If anyone has any questions from the slides or scripts feel free to drop me a line.   Pictures SharePoint Saturday Charlotte Apr '10 Pictures on Facebook (recommend these with comments and tagging)   View Full Album   Slides, Scripts, and Rating Links SharePoint Saturday Charlotte Apr '10 Slides and Demo Scripts SpeakerRate: The Power of PowerShell + SharePoint 2007 SpeakerRate: Managing SharePoint 2010 Farms with PowerShell   Conclusion     Big thanks out to Brian Gough (@bkgough), Dan Lewis (@sharepointcomic) and all of the other organizers of this event.  Also a big thanks out to the other speakers and sponsors (too many to list) who made the event possible.  Lastly thanks to my Sogeti coworker Kelly Jones (@kellydjones) for picking me up from the airport and a ride back to Columbus.  I hope everyone that attended got something out of the event and will continue to grow the SharePoint community.  I’m on a break from conferences for a few weeks and then have 3 more back to back weekends in May, blog posts announcing those coming later.  Enjoy the slides, scripts, and pictures.         -Frog Out

  Read the article

• Annoying mistake when create web template in SharePoint 2010

  - by ybbest

  I get the error    Error occurred in deployment step ‘Add Solution’: Exception from HRESULT: 0x8107026E when deploying my web template project. It turns out that your name for the Web template name has to match the name of you WebTemplate section in the element.xml file. Please see the screenshot below, the highlighted two needs to be the same. It took me so long to figure out and I will keep here in case everybody else struggle. If you are a newbie with web template in SharePoint 2010, this blog post explains the details.

  Read the article

• How to use SharePoint modal dialog box to display Custom Page Part3

  - by ybbest

  In the second part of the series, I showed you how to display and close a custom page in a SharePoint modal dialog using JavaScript and display a message after the modal dialog is closed. In this post, I’d like to show you how to use SPLongOperation with the Modal dialog box. You can download the source code here. 1. Firstly, modify the element file as follow   2. In your code behind, you can implement a close dialog function as below. This will close your modal dialog box once the button is clicked and display a status bar. Note that you need to use window.frameElement.commonModalDialogClose instead of window.frameElement.commonModalDialogClose References: How to: Display a Page as a Modal Dialog Box

  Read the article

• Using MVVM with Office365 and SharePoint 2010 REST API

  - by Sahil Malik

  SharePoint 2010 Training: more information I love JavaScript – people had pronounced this language dead a long time ago. But just like a chicken – which you eat before it’s born and after it’s dead, JavaScript – is being eaten all over the technical world, long after it’s dead! How nice! The coolest thing about JavaScript is that, There is no need for separate ActiveX controls, it is part of HTML/Browser It can interact with other DOM elements very very naturally It’s safe. And  it’s backwards and future compliant. It is no surprise thus that a number of libraries have emerged helping us work with JavaScript. But, JavaScript is not like C#. Notably, it has some biggies missing. For instance, Read full article ....

  Read the article

• SharePoint Unit Testing and Load Testing Finally?

  - by Kit Ong

  It has always been a real pain to incorporate extensive SharePoint Unit Testing and Load Testing in a project, could Visual Studio 2012 finally make this easier? It certaining looks like it, here's a brief overview on SharePoint support in Visual Studio 2012. Load testing – We now support load testing for SharePoint out of the box. This is more involved than you might imagine due to how dynamic SharePoint is. You can’t just record a script and play it back – it won’t work because SharePoint generates and expects dynamic data (like GUIDs). We’ve built the extensions to our load testing solution to parse the dynamic SharePoint data and include it appropriately in subsequent requests. So now you can record a script and play it back and we will dynamically adjust it to match what SharePoint expects.Unit testing – One of the big problems with unit testing SharePoint is that most code requires SharePoint to be running and trying to run tests against a live SharePoint instance is a pain. So we’ve built a SharePoint “emulator” using our new VS 2012 Fakes & Stubs capability. This will make unit testing of SharePoint components WAY easier.Read more in the link belowhttp://blogs.msdn.com/b/bharry/archive/2012/09/12/visual-studio-update-this-fall.aspx

  Read the article

• Workflow workarounds: tracking individual column changes

  - by PeterBrunone

  This post is long overdue, but since the question keeps popping up on various SharePoint discussion lists, I figured I'd document the answer here (next time I can just post a link instead of typing the whole thing out again).In short, you cannot trigger a SharePoint workflow when a column changes; you can only use the ItemChanged event.  To get more granular, then, you need to add some extra bits.Let's say you have a list called "5K Races" with a column called StartTime, and you want to execute some actions when the StartTime value changes.  Simply perform the following steps:1)  Create an additional column (same datatype) called OldStartTime.2)  When the workflow starts, compare StartTime to OldStartTime.    a) If they are equal, then do nothing (end).    b) If they are NOT equal, proceed with your workflow.3)  If 2b, then set OldStartTime to the value of StartTime.By performing step 3, you ensure that by the end of the workflow, OldStartTime will be equal to StartTime -- this is important because the workflow will continue to run every time a particular item is changed, but by taking away the criterion that would cause the workflow to run the second time, you have avoided an endless loop situation. 

  Read the article

• Ill be Speaking at ILTAs SharePoint for Legal Symposium on June 16th 2010

  Ill be speaking at the International Legal Technology Associations SharePoint for Legal Symposium on June 16th 2010 at Microsofts offices in Downers Grove, IL.  My talk will be about Building Public-Facing Websites with SharePoint 2010.  SharePoint has quickly become a popular platform for companies to build their public-facing websites on.  Ill go over the new features in SharePoint 2010 specific to web content management, and also discuss some best practices and lessons learned from our experience building internet sites with SharePoint. The SharePoint for Legal Symposium is a two-day event with talks covering a variety of other topics such as: Enterprise Search Using SharePoint 2010 and FAST SharePoint as a Document Management System Content Classification in SharePoint 2010: Taxonomies, Folksonomies and More Im very interested in hearing from firms who have been testing SharePoint 2010 prior to RTM, particularly how they are taking advantage of the new features in SharePoint 2010, e.g. Managed Metadata. Ive made my presentation available in advance, check it out on SlideShare: ILTA Presentation - Building Public-Facing Websites with SharePoint 2010 View more presentations from gdurzi. Did you know that DotNetSlackers also publishes .net articles written by top known .net Authors? We already have over 80 articles in several categories including Silverlight. Take a look: here.

  Read the article

• Getting the URL to the Content Type Hub Programmatically in SharePoint 2010

  - by Damon

  Many organizations use the content-type hub to manage content-types in their SharePoint 2010 environment.  As a developer in these types of organizations, you may one day find yourself in need of getting the URL of the content type hub programmatically.  Here is a quick snippet that demonstrates how to do it fairly painlessly: public static Uri GetContentTypeHubUri(SPSite site) {     TaxonomySession session = new TaxonomySession(site);     return Session.DefaultSiteCollectionTermStore         .ContentTypePublishingHub; }

  Read the article

• Big Data – Buzz Words: What is HDFS – Day 8 of 21

  - by Pinal Dave

  In yesterday’s blog post we learned what is MapReduce. In this article we will take a quick look at one of the four most important buzz words which goes around Big Data – HDFS. What is HDFS ? HDFS stands for Hadoop Distributed File System and it is a primary storage system used by Hadoop. It provides high performance access to data across Hadoop clusters. It is usually deployed on low-cost commodity hardware. In commodity hardware deployment server failures are very common. Due to the same reason HDFS is built to have high fault tolerance. The data transfer rate between compute nodes in HDFS is very high, which leads to reduced risk of failure. HDFS creates smaller pieces of the big data and distributes it on different nodes. It also copies each smaller piece to multiple times on different nodes. Hence when any node with the data crashes the system is automatically able to use the data from a different node and continue the process. This is the key feature of the HDFS system. Architecture of HDFS The architecture of the HDFS is master/slave architecture. An HDFS cluster always consists of single NameNode. This single NameNode is a master server and it manages the file system as well regulates access to various files. In additional to NameNode there are multiple DataNodes. There is always one DataNode for each data server. In HDFS a big file is split into one or more blocks and those blocks are stored in a set of DataNodes. The primary task of the NameNode is to open, close or rename files and directory and regulate access to the file system, whereas the primary task of the DataNode is read and write to the file systems. DataNode is also responsible for the creation, deletion or replication of the data based on the instruction from NameNode. In reality, NameNode and DataNode are software designed to run on commodity machine build in Java language. Visual Representation of HDFS Architecture Let us understand how HDFS works with the help of the diagram. Client APP or HDFS Client connects to NameSpace as well as DataNode. Client App access to the DataNode is regulated by NameSpace Node. NameSpace Node allows Client App to connect to the DataNode based by allowing the connection to the DataNode directly. A big data file is divided into multiple data blocks (let us assume that those data chunks are A,B,C and D. Client App will later on write data blocks directly to the DataNode. Client App does not have to directly write to all the node. It just has to write to any one of the node and NameNode will decide on which other DataNode it will have to replicate the data. In our example Client App directly writes to DataNode 1 and detained 3. However, data chunks are automatically replicated to other nodes. All the information like in which DataNode which data block is placed is written back to NameNode. High Availability During Disaster Now as multiple DataNode have same data blocks in the case of any DataNode which faces the disaster, the entire process will continue as other DataNode will assume the role to serve the specific data block which was on the failed node. This system provides very high tolerance to disaster and provides high availability. If you notice there is only single NameNode in our architecture. If that node fails our entire Hadoop Application will stop performing as it is a single node where we store all the metadata. As this node is very critical, it is usually replicated on another clustered as well as on another data rack. Though, that replicated node is not operational in architecture, it has all the necessary data to perform the task of the NameNode in the case of the NameNode fails. The entire Hadoop architecture is built to function smoothly even there are node failures or hardware malfunction. It is built on the simple concept that data is so big it is impossible to have come up with a single piece of the hardware which can manage it properly. We need lots of commodity (cheap) hardware to manage our big data and hardware failure is part of the commodity servers. To reduce the impact of hardware failure Hadoop architecture is built to overcome the limitation of the non-functioning hardware. Tomorrow In tomorrow’s blog post we will discuss the importance of the relational database in Big Data. Reference: Pinal Dave (http://blog.sqlauthority.com) Filed under: Big Data, PostADay, SQL, SQL Authority, SQL Query, SQL Server, SQL Tips and Tricks, T SQL

  Read the article

• List<T> paging asp.net

  - by user1397978

  Using a three-tier architecture, I have a list of objects List<object> careerList = new List<object>(); ModuleDTO module = new ModuleDTO(); careerList = module.getDegreeCodeByQualification(qualificationCode); which I then add to a gridview like so: gridViewMaster.DataSource = careerList; gridViewMaster.DataBind(); What I'd like to do is then enable paging on the gridview. My gridview so far is: <asp:GridView ID="gridViewMaster" runat="server" AutoGenerateColumns="False" GridLines="None" BorderWidth="1px" CellPadding="2" DataKeyNames="Grouping" ForeColor="Black" onrowdatabound="gridViewMaster_RowDataBound" CssClass="mGrid" PagerStyle-CssClass="pgr" AlternatingRowStyle-CssClass="alt" OnPageIndexChanging="gridView_PageIndexChanging" AllowPaging="True" > Is it possible to do enable paging on a list's without having to change that list to a Datatable or Dataview? If there is a way, this would help a lot. So far my events are as follows: protected void gridView_PageIndexChanging(object sender, GridViewPageEventArgs e) { gridViewMaster.PageIndex = e.NewPageIndex; List<object> careerList = new List<object>(); ModuleDTO module = new ModuleDTO(); careerList = module.getDegreeCodeByQualification(qualificationCode); ModalProgress.Show(); System.Threading.Thread.Sleep(1000); JobPanel.Visible = true; gridViewMaster.DataSource = careerList.Distinct(); gridViewMaster.DataBind(); } Someone PLEASE HELP ME!!! Thank you

  Read the article

• Oracle Data Integration 12c: Perspectives of Industry Experts, Customers and Partners

  - by Irem Radzik

  Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 As you may have seen from our recent blog posts on Oracle Data Integrator 12c and Oracle GoldenGate 12c, we are very excited to share with you the great new features the 12c release brings to Oracle’s data integration solutions. And, fortunately we are not alone in this sentiment. Since the press announcement October 17th, which incorporates our customers' and experts' testimonials, we have seen positive comments in leading technology publications and social media as well. Here are some examples: In CIO and PCWorld you can find Joab Jackson’s article, Oracle Data Integrator 12c ready for real-time analysis, where wrote about the tight integration between Oracle Data Integrator and Oracle GoldenGate . He noted “Heeding the call from enterprise customers who clamor for more immediacy in their data-driven reports, Oracle has updated its data-integration software portfolio so that it can more rapidly deliver data to data warehouses and analysis applications.” Integration Developer News’ Vance McCarthy wrote the article Oracle Ships ‘Future Proofs’ Integration Tools for Traditional, Cloud, Big Data, Real-Time Projects and mentioned that “Oracle Data Integrator 12c and Oracle GoldenGate 12c sport a wide range of improvements to let devs more easily deliver data integration for cloud, analytics, big data and other new projects that leverage multiple datasets for business.“ InformationWeek’s Doug Henschen gave a great overview to several key features including the new flow-based UI in Oracle Data Integrator. Doug said “Oracle Data Integrator 12c introduces a complete makeover of the job-building experience, while real-time oriented GoldenGate 12c introduces performance gains “. In Database Trends and Applications’ article Oracle Strengthens Data Integration with Release of Oracle Data Integrator 12c and Oracle GoldenGate 12c highlighted the productivity aspect of the new solution with his remarks: “tight integration between Oracle Data Integrator 12c and Oracle GoldenGate 12c enables developers to leverage Oracle GoldenGate’s low overhead, real-time change data capture completely within the Oracle Data Integrator Studio without additional training”. We are also thrilled about what our customers and partners have to say about our products and the new release. And we are equally excited to share those perspectives with you in our upcoming launch video webcast on November 12th. SolarWorld Industries America’s Senior Database Manager, Russ Toyama will join our executives in our studio in Redwood Shores to discuss GoldenGate’s core benefits and the new release, while Surren Partharb, CTO of Strategic Technology Services for BT, and Mark Rittman, CTO of Rittman Mead, will provide their comments via the interviews conducted in the UK. This interactive panel discussion in the video webcast will unveil the new release with the expertise of our development executives and the great insight from our customers and partners. In addition, our product experts will be available online to answer chat questions. This is really a great opportunity to learn how Oracle's data integration offering has changed the integration and replication technology space with the new release, and established itself as the new leader. If you have not registered for this free event yet, you can do so via this link. We will run the live event at 8am PT/4pm GMT, followed by a replay of the event with live chat for Q&A  at 10am PT/6pm GMT. The replay will be available on-demand for those who register but cannot attend either session on November 12th. /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Times New Roman","serif"; mso-fareast-font-family:"Times New Roman";}

  Read the article

• Subset a data.frame by list and apply function on each part, by rows

  - by aL3xa

  This may seem as a typical plyr problem, but I have something different in mind. Here's the function that I want to optimize (skip the for loop). # dummy data set.seed(1985) lst <- list(a=1:10, b=11:15, c=16:20) m <- matrix(round(runif(200, 1, 7)), 10) m <- as.data.frame(m) dfsub <- function(dt, lst, fun) { # check whether dt is `data.frame` stopifnot (is.data.frame(dt)) # check if vectors in lst are "whole" / integer # vector elements should be column indexes is.wholenumber <- function(x, tol = .Machine$double.eps^0.5) abs(x - round(x)) < tol # fall if any non-integers in list idx <- rapply(lst, is.wholenumber) stopifnot(idx) # check for list length stopifnot(ncol(dt) == length(idx)) # subset the data subs <- list() for (i in 1:length(lst)) { # apply function on each part, by row subs[[i]] <- apply(dt[ , lst[[i]]], 1, fun) } # preserve names names(subs) <- names(lst) # convert to data.frame subs <- as.data.frame(subs) # guess what =) return(subs) } And now a short demonstration... actually, I'm about to explain what I primarily intended to do. I wanted to subset a data.frame by vectors gathered in list object. Since this is a part of code from a function that accompanies data manipulation in psychological research, you can consider m as a results from personality questionnaire (10 subjects, 20 vars). Vectors in list hold column indexes that define questionnaire subscales (e.g. personality traits). Each subscale is defined by several items (columns in data.frame). If we presuppose that the score on each subscale is nothing more than sum (or some other function) of row values (results on that part of questionnaire for each subject), you could run: > dfsub(m, lst, sum) a b c 1 46 20 24 2 41 24 21 3 41 13 12 4 37 14 18 5 57 18 25 6 27 18 18 7 28 17 20 8 31 18 23 9 38 14 15 10 41 14 22 I took a glance at this function and I must admit that this little loop isn't spoiling the code at all... BUT, if there's an easier/efficient way of doing this, please, let me know!

  Read the article

• Copy folders when copying list items from source to destination

  - by iHeartDucks

  Hi, This is my code to copy files in a list from source to destination. Using the code below I am only able to copy files but not folders. Any ideas on how can I copy the folders and the files within those folders? using (SPSite objSite = new SPSite(URL)) { using (SPWeb objWeb = objSite.OpenWeb()) { SPList objSourceList = null; SPList objDestinationList = null; try { objSourceList = objWeb.Lists["Source"]; } catch(Exception ex) { Console.WriteLine("Error opening source list"); Console.WriteLine(ex.Message); } try { objDestinationList = objWeb.Lists["Destination"]; } catch (Exception ex) { Console.WriteLine("Error opening destination list"); Console.WriteLine(ex.Message); } string ItemURL = string.Empty; if (objSourceList != null && objDestinationList != null) { foreach (SPListItem objSourceItem in objSourceList.Items) { ItemURL = string.Format(@"{0}/Destination/{1}", objDestinationList.ParentWeb.Url, objSourceItem.Name); objSourceItem.CopyTo(ItemURL); objSourceItem.UnlinkFromCopySource(); } } } } Thanks

  Read the article

• New Big Data Appliance Security Features

  - by mgubar

  The Oracle Big Data Appliance (BDA) is an engineered system for big data processing.  It greatly simplifies the deployment of an optimized Hadoop Cluster – whether that cluster is used for batch or real-time processing.  The vast majority of BDA customers are integrating the appliance with their Oracle Databases and they have certain expectations – especially around security.  Oracle Database customers have benefited from a rich set of security features:  encryption, redaction, data masking, database firewall, label based access control – and much, much more.  They want similar capabilities with their Hadoop cluster.    Unfortunately, Hadoop wasn’t developed with security in mind.  By default, a Hadoop cluster is insecure – the antithesis of an Oracle Database.  Some critical security features have been implemented – but even those capabilities are arduous to setup and configure.  Oracle believes that a key element of an optimized appliance is that its data should be secure.  Therefore, by default the BDA delivers the “AAA of security”: authentication, authorization and auditing. Security Starts at Authentication A successful security strategy is predicated on strong authentication – for both users and software services.  Consider the default configuration for a newly installed Oracle Database; it’s been a long time since you had a legitimate chance at accessing the database using the credentials “system/manager” or “scott/tiger”.  The default Oracle Database policy is to lock accounts thereby restricting access; administrators must consciously grant access to users. Default Authentication in Hadoop By default, a Hadoop cluster fails the authentication test. For example, it is easy for a malicious user to masquerade as any other user on the system.  Consider the following scenario that illustrates how a user can access any data on a Hadoop cluster by masquerading as a more privileged user.  In our scenario, the Hadoop cluster contains sensitive salary information in the file /user/hrdata/salaries.txt.  When logged in as the hr user, you can see the following files.  Notice, we’re using the Hadoop command line utilities for accessing the data: $ hadoop fs -ls /user/hrdataFound 1 items-rw-r--r--   1 oracle supergroup         70 2013-10-31 10:38 /user/hrdata/salaries.txt$ hadoop fs -cat /user/hrdata/salaries.txtTom Brady,11000000Tom Hanks,5000000Bob Smith,250000Oprah,300000000 User DrEvil has access to the cluster – and can see that there is an interesting folder called “hrdata”.  $ hadoop fs -ls /user Found 1 items drwx------   - hr supergroup          0 2013-10-31 10:38 /user/hrdata However, DrEvil cannot view the contents of the folder due to lack of access privileges: $ hadoop fs -ls /user/hrdata ls: Permission denied: user=drevil, access=READ_EXECUTE, inode="/user/hrdata":oracle:supergroup:drwx------ Accessing this data will not be a problem for DrEvil. He knows that the hr user owns the data by looking at the folder’s ACLs. To overcome this challenge, he will simply masquerade as the hr user. On his local machine, he adds the hr user, assigns that user a password, and then accesses the data on the Hadoop cluster: $ sudo useradd hr $ sudo passwd $ su hr $ hadoop fs -cat /user/hrdata/salaries.txt Tom Brady,11000000 Tom Hanks,5000000 Bob Smith,250000 Oprah,300000000 Hadoop has not authenticated the user; it trusts that the identity that has been presented is indeed the hr user. Therefore, sensitive data has been easily compromised. Clearly, the default security policy is inappropriate and dangerous to many organizations storing critical data in HDFS. Big Data Appliance Provides Secure Authentication The BDA provides secure authentication to the Hadoop cluster by default – preventing the type of masquerading described above. It accomplishes this thru Kerberos integration. Figure 1: Kerberos Integration The Key Distribution Center (KDC) is a server that has two components: an authentication server and a ticket granting service. The authentication server validates the identity of the user and service. Once authenticated, a client must request a ticket from the ticket granting service – allowing it to access the BDA’s NameNode, JobTracker, etc. At installation, you simply point the BDA to an external KDC or automatically install a highly available KDC on the BDA itself. Kerberos will then provide strong authentication for not just the end user – but also for important Hadoop services running on the appliance. You can now guarantee that users are who they claim to be – and rogue services (like fake data nodes) are not added to the system. It is common for organizations to want to leverage existing LDAP servers for common user and group management. Kerberos integrates with LDAP servers – allowing the principals and encryption keys to be stored in the common repository. This simplifies the deployment and administration of the secure environment. Authorize Access to Sensitive Data Kerberos-based authentication ensures secure access to the system and the establishment of a trusted identity – a prerequisite for any authorization scheme. Once this identity is established, you need to authorize access to the data. HDFS will authorize access to files using ACLs with the authorization specification applied using classic Linux-style commands like chmod and chown (e.g. hadoop fs -chown oracle:oracle /user/hrdata changes the ownership of the /user/hrdata folder to oracle). Authorization is applied at the user or group level – utilizing group membership found in the Linux environment (i.e. /etc/group) or in the LDAP server. For SQL-based data stores – like Hive and Impala – finer grained access control is required. Access to databases, tables, columns, etc. must be controlled. And, you want to leverage roles to facilitate administration. Apache Sentry is a new project that delivers fine grained access control; both Cloudera and Oracle are the project’s founding members. Sentry satisfies the following three authorization requirements: Secure Authorization:  the ability to control access to data and/or privileges on data for authenticated users. Fine-Grained Authorization:  the ability to give users access to a subset of the data (e.g. column) in a database Role-Based Authorization:  the ability to create/apply template-based privileges based on functional roles. With Sentry, “all”, “select” or “insert” privileges are granted to an object. The descendants of that object automatically inherit that privilege. A collection of privileges across many objects may be aggregated into a role – and users/groups are then assigned that role. This leads to simplified administration of security across the system. Figure 2: Object Hierarchy – granting a privilege on the database object will be inherited by its tables and views. Sentry is currently used by both Hive and Impala – but it is a framework that other data sources can leverage when offering fine-grained authorization. For example, one can expect Sentry to deliver authorization capabilities to Cloudera Search in the near future. Audit Hadoop Cluster Activity Auditing is a critical component to a secure system and is oftentimes required for SOX, PCI and other regulations. The BDA integrates with Oracle Audit Vault and Database Firewall – tracking different types of activity taking place on the cluster: Figure 3: Monitored Hadoop services. At the lowest level, every operation that accesses data in HDFS is captured. The HDFS audit log identifies the user who accessed the file, the time that file was accessed, the type of access (read, write, delete, list, etc.) and whether or not that file access was successful. The other auditing features include: MapReduce:  correlate the MapReduce job that accessed the file Oozie:  describes who ran what as part of a workflow Hive:  captures changes were made to the Hive metadata The audit data is captured in the Audit Vault Server – which integrates audit activity from a variety of sources, adding databases (Oracle, DB2, SQL Server) and operating systems to activity from the BDA. Figure 4: Consolidated audit data across the enterprise.  Once the data is in the Audit Vault server, you can leverage a rich set of prebuilt and custom reports to monitor all the activity in the enterprise. In addition, alerts may be defined to trigger violations of audit policies. Conclusion Security cannot be considered an afterthought in big data deployments. Across most organizations, Hadoop is managing sensitive data that must be protected; it is not simply crunching publicly available information used for search applications. The BDA provides a strong security foundation – ensuring users are only allowed to view authorized data and that data access is audited in a consolidated framework.

  Read the article

• How to present a stable data model in a public API that allows internal data structures to be changed without breaking the public view of the data?

  - by Max Palmer

  I am in the process of developing an application that allows users to write C# scripts. These scripts allow users to call selected methods and to access and manipulate data in a document. This works well, however, in the development version, scripts access the document's (internal) data structures directly. This means that if we were to change the internal data model/structure, there is a good chance that someone's script will no longer compile. We obviously want to prevent this breaking change from happening, but still want to allow the user to write sensible C# code (whilst not restricting how we develop our internal data model as a result). We therefore need to decouple our scripting API and its data structures from our internal methods and data structures. We've a few ideas as to how we might allow the user to access a what is effectively a stable public version of the document's internal data*, but I wanted to throw the question out there to someone who might have some real experience of this problem. NB our internal document's data structure is quite complex and it could be quite difficult to wrap. We know we want to expose as little as possible in our public API, especially as once it's out there, it's out there for good. Can anyone help? How do scripting languages / APIs decouple their public API and data structures from their internal data structures? Is there no real alternative to having to write a complex interaction layer? If we need to do this, what's a good approach or pattern for wrapping complex data structures that include nested objects, including collections? I've looked at the API facade pattern, which looks like it's trying to address these kinds of issues, but are there alternatives? *One idea is to build a data facade that is kept stable across versions of our application. The facade exposes a set of facade data objects that are used in the script code. These maintain backwards compatibility and wrap access to our internal document's data model.

  Read the article

• Python 2.7 list of lists manipulation functionality

  - by user3688163

  I am trying to perform several operations on the myList list of lists below and am having some trouble figuring it out. I am very new to Python. myList = [ ['Issue Id','1.Completeness for OTC','Break',3275,33,33725102303,296384802,20140107], ['Issue Id','2.Validity check1 for OTC','Break',3308,0,34021487105,0,20140107], ['Issue Id','3.Validity check2 for OTC','Break',3308,0,34021487105,0,20140107], ['Issue Id','4.Completeness for RST','Break',73376,1,8.24931E+11,44690130,20140107], ['Issue Id','5.Validity check1 for RST','Break',73377,0,8.24976E+11,0,20140107], ['Liquidity','1. OTC - Null','Break',7821 0,2.28291E+11,0,20140110], ['Liquidity','2. OTC - Unmapped','Break',7778,43,2.27712E+11,579021732.8,20140110], ['Liquidity','3. RST - Null','Break',335120,0,1.01425E+12,0,20140110], ['Liquidity','4. RST - Unmapped','Break',334608,512,1.01351E+12,735465433.1,20140110], ['Liquidity','5. RST - Valid','Break',335120,0,1.01425E+12,0,20140110], ['Issue Id','1.Completeness for OTC','Break',3292,33,32397924450,306203929,20140110], ['Issue Id','2.Validity check1 for OTC','Break',3325,0,32704128379,0,20140110], ['Issue Id','3.Validity check2 for OTC','Break',3325,0,32704128379,0,20140110], ['Issue Id','4.Completeness for RST','Break',73594,3,8.5352E+11,69614602,20140110], ['Issue Id','5.Validity check1 for RST','Break',73597,0,8.5359E+11,0,20140110], ['Unlinked Silver ID','DQ','Break',3201318,176,20000000,54974.33386,20140101], ['Missing GCI','DQ','Break',3201336,158,68000000,49351.9588,20140101], ['Missing Book','DQ Break',3192720,8774,3001000000,2740595.484,20140101], ['Matured Trades','DQ','Break',3201006,488,1371000000,152428.8348,20140101], ['Illiquid Trades','1.Completeness Check for range','Break',43122,47,88597695671,54399061.43,20140107], ['Illiquid Trades','2.Completeness Check for non','Break',39033,0,79133622401,0,20140107] ] I am trying to get the result below but do not know how to do so: newList = [ ['Issue Id','1.Completeness for OTC:2.Validity check1 for OTC:3.Validity check2 for OTC','Break',3275,33,33725102303,296384802,20140107], ['Issue Id','4.Completeness for RST:5.Validity check1 for RST','Break',73376,1,8.24931E+11,44690130,20140107], ['Liquidity','1. OTC - Null','Break:2. OTC - Unmapped','Break',7821 0,2.28291E+11,0,20140110], ['Liquidity','3. RST - Null:4. RST - Unmapped:5. RST - Valid','Break',335120,0,1.01425E+12,0,20140110], ['Issue Id','1.Completeness for OTC:2.Validity check1 for OTC:3.Validity check2 for OTC','Break',3292,33,32397924450,306203929,20140110], ['Issue Id','4.Completeness for RST:5. RST - Valid','Break',73594,3,8.5352E+11,69614602,20140110], ['Unlinked Silver ID','DQ','Break',3201318,176,20000000,54974.33386,20140101], ['Missing GCI','DQ','Break',3201336,158,68000000,49351.9588,20140101], ['Missing Book','DQ Break',3192720,8774,3001000000,2740595.484,20140101], ['Matured Trades','DQ','Break',3201006,488,1371000000,152428.8348,20140101], ['Illiquid Trades','1.Completeness Check for range','Break',43122,47,88597695671,54399061.43,20140107], ['Illiquid Trades','2.Completeness Check for non','Break',39033,0,79133622401,0,20140107] ] Rules to create newList. Create a new list in the newList list of lists if the values in the lists meet the following conditions: multiple lists that match on `myList[i][0]` and `myList[i][7]` but with have (1) sums of `myList[i][3]` and `myList[i][4]` and (2) sums of `myList[i][5]` and `myList[i][6]` that are different from each other are just listed as is in newList if multiple lists match on both `myList[i][0]` (this is the type) and `myList[i][7]` (this is the date) are the same then create a new list for each set of lists with mathcing `myList[i][0]` and `myList[i][7]` that have (1) sums of `myList[i][3]` and `myList[i][4]` and (2) sums of `myList[i][5]` and `myList[i][6]` that are different from the other lists with mathcing `myList[i][0]` and `myList[i][7]`. I also am trying to concatenate `myList[i][1]` separated by a ':' for all those lists with matching `myList[i][0]` and `myList[i][7]` and with sums of `myList[i][3]` + `myList[i][4]` and `myList[i][5]` + `myList[i][6]` that match. So essentially for this case only those lists in `myList` with sums of `myList[i][3]` + `myList[i][4]` and `myList[i][5]` + `myList[i][6]` are different from the other lists are then listed in newList. The above newList illustrates these results I am trying to achieve. If anyone has any ideas how to do this they would be greatly appreciated. Thank you!

  Read the article

• How to start Sharepoint Development - Resources wanted [closed]

  - by user1249641

  I'm a apprentice for software development here in good ol' Germany and I've been doing fairly well with c#, asp.net mvc, entity framework and LINQ so far. My superiors want me to focus solely on our intranet development and Sharepoint development. They don't provide me with any resources to start. No books, no co-workers with actual webpart-dev-experience, seminars and the likes. There for it's do it on your own or die. I bought a book and started working through it on my virtual machine messing with the infrastructure and everything i can get a grip on. My main problem however stays the actual development. I have managed to write 2 webparts which can be used as a rudimentary ticket system(using WSP Builder and SP2007). But there it stops. Are there any comprehensive step by step tutorials or blogs out there, like the asp.net tutorials on www.asp.net which take you by the hand go over each step with you? Starting with the basic classes, going over custom css implementation, Jquery/Javascript ajax and async calls? No matter how trivial, I appreciate every help and hint you can give.

  Read the article

• SharePoint 2010 Data Retrival Techinques

  - by Jayant Sharma

  In SharePoint, we have two options to perform CRUD operation.1. using server side code2. using client side codeusing server side code, we have 1. CAML2. LINQusing client side code, we have 1. Client Object Model    1.1.      Managed Client Object Model     1.2.     Silverlight Client Object Model    1.3.     ECMA Client Object Model2. SharePoint Web Services3. ADO Data Service (based on REST Web Services)4. Using RPC Call (owssvr.dll)Which and when these options are used depend upon requirements. Every options are certain advantages and disadvantages. So, before start development of any new sharepoint project, it is important to understand the limitations of different methods.Server Object Model is used when our application is host on the same server on which sharepoint is installed. while Client Side code is used to access sharepoint from client system. In SharePoint 2010 specially Client Object Model (COM) are introduced to perform the sharepoint operations from client system. Advantage of CAML:    -  It is fast.    -  Can be use it from all kind of technology like Silverlight, or Jquery    -  You can use U2U CAML Query builder to generate CAML Query.Disadvantage Of CAML:    - Error Prone, as we can detect the error only at runtimeAdvantage of LINQ:    -  Object Oriented technique (Object Relation Model)    -  LINQ  to SharePoint provider are working with Strongly Type List Item Objects, So intellisence are present at runtime    -  No need of knowledge of CAML    -  Less Error Prone as it as it uses C# syntex.    -  You can compare two Fields of SharePoint ListDisadvantage Of LINQ:    -  List Attachment is not supported in SPMetal Tool    -  Created By, Created, Modified and Modified By Fields are not created by SPMetal Tool.    -  Custom fields are not created by SPMetal Tools    -  External Lists are not supported    -  Though at backend LINQ genenates CAML Query so it is slower than directly using CAML in Code.  Advantage of Client Object Model    -  Used to access sharepoint from client system    -  No WebServer is required at Client End    - Can use Silverlight and JavaScripts to make better and fast User experienceDisadvantage of Client Object Model    -  You cannot use RunwithEleveatedPrivilege    - Cross Site Collection query are not possible    - Lesser API's are availableADO.Net Data Services:    -  Only List based operations are possible, other type of operations are not possible.SharePoint Web Services and RPC Call:    - Previously it was used in SharePoint 2007 but after the introduction  of Client Object Model,  Microsoft recommends not to use Web Services to fetch data from SharePoint. In SharePoint 2010 it is avaliable only for backward compatibility.Ref: http://msdn.microsoft.com/en-us/library/ee539764Jayant Sharma

  Read the article

• How Do I Restrict Views of a Custom List by Group in Sharepoint 2007?

  - by Crash893

  I'm pretty new to Sharepoint and what I would like to do is create a huge master list of all our employees and then make different "views" on that person depending on the persons group For example: A new employee might have Salary info Security info Personnel info Contract info I would like to have all that in one row (per employee) but then when someone from the hr group logs in they can only see Personnel and Salary or something like that. If that is not an option is there a way to link tables across different lists?

  Read the article

< Previous Page | 14 15 16 17 18 19 20 21 22 23 24 25  | Next Page >