Search Results

Search found 2088 results on 84 pages for 'jobs'.

Page 21/84 | < Previous Page | 17 18 19 20 21 22 23 24 25 26 27 28  | Next Page >

• Job queueing and execute Mechanism

  - by Calm Storm

  In my webservice all method calls submits jobs to a queue. Basically these operations take long time to execute, so all these operations submit a Job to a queue and return a status saying "Submitted". Then the client keeps polling using another service method to check for the status of the job. Presently, what I do is create my own Queue, Job classes that are Serializable and persist these jobs (i.e, their serialized byte stream format) into the database. So an UpdateLogistics operation just queues up a "UpdateLogisticsJob" to the queue and returns. I have written my own JobExecutor which wakes up every N seconds, scans the database table for any existing jobs, and executes them. Note the jobs have to persisted because these jobs have to survive app-server crashes. This was done a long time ago, and I used bespoke classes for my Queues, Jobs, Executors etc. But now, I would like to know has someone done something similar before? In particular, Are there frameworks available for this ? Something in Spring/Apache etc Any framework that is easy to adapt/debug and plays well along with libraries like Spring will be great.

  Read the article

• Algorithm to retrieve every possible combination of sublists of a two lists

  - by sgmoore

  Suppose I have two lists, how do I iterate through every possible combination of every sublist, such that each item appears once and only once. I guess an example could be if you have employees and jobs and you want split them into teams, where each employee can only be in one team and each job can only be in one team. Eg List<string> employees = new List<string>() { "Adam", "Bob"} ; List<string> jobs = new List<string>() { "1", "2", "3"}; I want Adam : 1 Bob : 2 , 3 Adam : 1 , 2 Bob : 3 Adam : 1 , 3 Bob : 2 Adam : 2 Bob : 1 , 3 Adam : 2 , 3 Bob : 1 Adam : 3 Bob : 1 , 2 Adam, Bob : 1, 2, 3 I tried using the answer to this stackoverflow question to generate a list of every possible combination of employees and every possible combination of jobs and then select one item from each from each list, but that's about as far as I got. I don't know the maximum size of the lists, but it would be certainly be less than 100 and there may be other limiting factors (such as each team can have no more than 5 employees) Update Not sure whether this can be tidied up more and/or simplified, but this is what I have ended up with so far. It uses the Group algorithm supplied by Yorye (see his answer below), but I removed the orderby which I don't need and caused problems if the keys are not comparable. var employees = new List<string>() { "Adam", "Bob" } ; var jobs = new List<string>() { "1", "2", "3" }; int c= 0; foreach (int noOfTeams in Enumerable.Range(1, employees.Count)) { var hs = new HashSet<string>(); foreach( var grouping in Group(Enumerable.Range(1, noOfTeams).ToList(), employees)) { // Generate a unique key for each group to detect duplicates. var key = string.Join(":" , grouping.Select(sub => string.Join(",", sub))); if (!hs.Add(key)) continue; List<List<string>> teams = (from r in grouping select r.ToList()).ToList(); foreach (var group in Group(teams, jobs)) { foreach (var sub in group) { Console.WriteLine(String.Join(", " , sub.Key ) + " : " + string.Join(", ", sub)); } Console.WriteLine(); c++; } } } Console.WriteLine(String.Format("{0:n0} combinations for {1} employees and {2} jobs" , c , employees.Count, jobs.Count)); Since I'm not worried about the order of the results, this seems to give me what I need.

  Read the article

• SQL Server 2008 Recovery Mode reverts from FULL to SIMPLE

  - by Eric Hazen

  Three of our SQL databases have their recovery model change every night from FULL to SIMPLE. The only jobs that I'm aware of are two BackupExec jobs that run nightly. Why would the recovery model change? Backup Jobs: SQL FULL BACKUP, SQL LOG BACKUP Event Manager: Event 5084: Setting Database option RECOVERY to SIMPLE for database databaseName

  Read the article

• How do I configure the number of worker threads used by SqlServer 2005 Agent

  - by Decker

  How can I increase the limit of worker threads from the default 10 for SQL Server 2005 SqlAgent? I have 9 jobs that run almost continuously and that leaves only one available thread for the rest of the scheduled jobs. Oftentimes, when no thread is available, I will see the jobs in "Waiting for worker thread" state. I'd like to increase the number to about 12 (which should do the trick for me). Any idea where this is set?

  Read the article

• Linux buffer cache effect on IO writes?

  - by Patrick LeBoutillier

  I'm copying large files (3 x 30G) between 2 filesystems on a Linux server (kernel 2.6.37, 16 cores, 32G RAM) and I'm getting poor performance. I suspect that the usage of the buffer cache is killing the I/O performance. To try and narrow down the problem I used fio directly on the SAS disk to monitor the performance. Here is the output of 2 fio runs (the first with direct=1, the second one direct=0): Config: [test] rw=write blocksize=32k size=20G filename=/dev/sda # direct=1 Run 1: test: (g=0): rw=write, bs=32K-32K/32K-32K, ioengine=sync, iodepth=1 Starting 1 process Jobs: 1 (f=1): [W] [100.0% done] [0K/205M /s] [0/6K iops] [eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=4667 write: io=20,480MB, bw=199MB/s, iops=6,381, runt=102698msec clat (usec): min=104, max=13,388, avg=152.06, stdev=72.43 bw (KB/s) : min=192448, max=213824, per=100.01%, avg=204232.82, stdev=4084.67 cpu : usr=3.37%, sys=16.55%, ctx=655410, majf=0, minf=29 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued r/w: total=0/655360, short=0/0 lat (usec): 250=99.50%, 500=0.45%, 750=0.01%, 1000=0.01% lat (msec): 2=0.01%, 4=0.02%, 10=0.01%, 20=0.01% Run status group 0 (all jobs): WRITE: io=20,480MB, aggrb=199MB/s, minb=204MB/s, maxb=204MB/s, mint=102698msec, maxt=102698msec Disk stats (read/write): sda: ios=0/655238, merge=0/0, ticks=0/79552, in_queue=78640, util=76.55% Run 2: test: (g=0): rw=write, bs=32K-32K/32K-32K, ioengine=sync, iodepth=1 Starting 1 process Jobs: 1 (f=1): [W] [100.0% done] [0K/0K /s] [0/0 iops] [eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=4733 write: io=20,480MB, bw=91,265KB/s, iops=2,852, runt=229786msec clat (usec): min=16, max=127K, avg=349.53, stdev=4694.98 bw (KB/s) : min=56013, max=1390016, per=101.47%, avg=92607.31, stdev=167453.17 cpu : usr=0.41%, sys=6.93%, ctx=21128, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued r/w: total=0/655360, short=0/0 lat (usec): 20=5.53%, 50=93.89%, 100=0.02%, 250=0.01%, 500=0.01% lat (msec): 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.12% lat (msec): 100=0.38%, 250=0.04% Run status group 0 (all jobs): WRITE: io=20,480MB, aggrb=91,265KB/s, minb=93,455KB/s, maxb=93,455KB/s, mint=229786msec, maxt=229786msec Disk stats (read/write): sda: ios=8/79811, merge=7/7721388, ticks=9/32418456, in_queue=32471983, util=98.98% I'm not knowledgeable enough with fio to interpret the results, but I don't expect the overall performance using the buffer cache to be 50% less than with O_DIRECT. Can someone help me interpret the fio output? Are there any kernel tunings that could fix/minimize the problem? Thanks a lot,

  Read the article

• Manually start scheduled launchd job

  - by Pascal

  On our Mac OS X (10.6) Server we have setup several backup scripts that are controlled by launchd and launched at specific times. For this we have defined StartCalendarInterval and this all works very well. Now it happens that I would like to start one of these jobs out of schedule, but this does not start the job (but also does not give an error/warning): sudo launchctl start org.job-label The manpage of launchtl states that start is intended to test on-demand jobs, no word of scheduled jobs. Is there a way to kickstart scheduled jobs?

  Read the article

• Linux buffer cache effect on IO writes?

  - by Patrick LeBoutillier

  Hi, I'm copying large files (3 x 30G) between 2 filesystems on a Linux server (kernel 2.6.37, 16 cores, 32G RAM) and I'm getting poor performance. I suspect that the usage of the buffer cache is killing the I/O performance. To try and narrow down the problem I used fio directly on the SAS disk to monitor the performance. Here is the output of 2 fio runs (the first with direct=1, the second one direct=0): Config: [test] rw=write blocksize=32k size=20G filename=/dev/sda # direct=1 Run 1: test: (g=0): rw=write, bs=32K-32K/32K-32K, ioengine=sync, iodepth=1 Starting 1 process Jobs: 1 (f=1): [W] [100.0% done] [0K/205M /s] [0/6K iops] [eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=4667 write: io=20,480MB, bw=199MB/s, iops=6,381, runt=102698msec clat (usec): min=104, max=13,388, avg=152.06, stdev=72.43 bw (KB/s) : min=192448, max=213824, per=100.01%, avg=204232.82, stdev=4084.67 cpu : usr=3.37%, sys=16.55%, ctx=655410, majf=0, minf=29 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued r/w: total=0/655360, short=0/0 lat (usec): 250=99.50%, 500=0.45%, 750=0.01%, 1000=0.01% lat (msec): 2=0.01%, 4=0.02%, 10=0.01%, 20=0.01% Run status group 0 (all jobs): WRITE: io=20,480MB, aggrb=199MB/s, minb=204MB/s, maxb=204MB/s, mint=102698msec, maxt=102698msec Disk stats (read/write): sda: ios=0/655238, merge=0/0, ticks=0/79552, in_queue=78640, util=76.55% Run 2: test: (g=0): rw=write, bs=32K-32K/32K-32K, ioengine=sync, iodepth=1 Starting 1 process Jobs: 1 (f=1): [W] [100.0% done] [0K/0K /s] [0/0 iops] [eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=4733 write: io=20,480MB, bw=91,265KB/s, iops=2,852, runt=229786msec clat (usec): min=16, max=127K, avg=349.53, stdev=4694.98 bw (KB/s) : min=56013, max=1390016, per=101.47%, avg=92607.31, stdev=167453.17 cpu : usr=0.41%, sys=6.93%, ctx=21128, majf=0, minf=33 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued r/w: total=0/655360, short=0/0 lat (usec): 20=5.53%, 50=93.89%, 100=0.02%, 250=0.01%, 500=0.01% lat (msec): 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.12% lat (msec): 100=0.38%, 250=0.04% Run status group 0 (all jobs): WRITE: io=20,480MB, aggrb=91,265KB/s, minb=93,455KB/s, maxb=93,455KB/s, mint=229786msec, maxt=229786msec Disk stats (read/write): sda: ios=8/79811, merge=7/7721388, ticks=9/32418456, in_queue=32471983, util=98.98% I'm not knowledgeable enough with fio to interpret the results, but I don't expect the overall performance using the buffer cache to be 50% less than with O_DIRECT. Can someone help me interpret the fio output? Are there any kernel tunings that could fix/minimize the problem? Thanks a lot,

  Read the article

• Keep printed documents on Windows Server 2008 R2 Print Server

  - by MadBoy

  I've setup Windows 2008 R2 as print server. I have checked option Keep printed document option for all printers and it works fine. Users print their stuff and i can see what they are doing. Problem is everyone sees all documents that are getting printed which is not always the best idea. Is there a way to: Limit print jobs to be only seen by people who printed them and admins Limit print jobs to be only seen on server (from within Server Manager) and so print jobs dissapear when print job is done from user queue (but then admins are still able to see it and track what's printed and when for reporting purposes). Create some kind of access level list so that some people can see everything geting printed, some people see their print jobs and some people see nothing :-)

  Read the article

• BITS http download job fails to connect for owner Local SYSTEM account

  - by MikeT

  A service I have written that uses BITS (Background Intelligent Transfer Service) to auto update itself is having a problem on some machines (Windows 7 so far). I have been investigating and have discovered that some of the jobs that my service adds to the bits queue are failing immediately with the error code 0x80072efd (a connection with this server could not be established). The is not problem with connecting to the server for the download as it works fine on the same machine using IE (or any other web browser) and other clients can connect and update from the same server. I tried using the BITSADMIN.exe tool to add the jobs manually and they worked ok. I then changed the account my service was running under to the network service account so the bits jobs would be created with a different owner and the jobs completed successfully. My question is I don't want to run my service as this account as it wont have the required local permissions, so how to I change the permissions of the local system user to allow it to download from the HTTP source, I'm not aware of any way of this being restricted for this account but it obviously is.

  Read the article

• create a view to show the backup status in every 10 mins

  - by user2853141

  I have a question, my table have following data: userID, startTime, EndTime ————————————— 101, 04/11/2013 11:00:00, 04/11/2013 11:55:00 102, 04/11/2013 11:00:00, 04/11/2013 11:24:00 103, 04/11/2013 11:20:00, 04/11/2013 11:45:00 104, 04/11/2013 11:30:00, 04/11/2013 11:35:00 105, 04/11/2013 11:40:00, 04/11/2013 11:55:00 can I use the view to show the backup status in every 10 mins? I wonder the result as following: time, count —————————— 04/11/2013 11:00:00, 2 04/11/2013 11:10:00, 2 04/11/2013 11:20:00, 3 04/11/2013 11:30:00, 3 04/11/2013 11:40:00, 3 04/11/2013 11:50:00, 2 04/11/2013 12:00:00, 0 04/11/2013 11:00:00 – 04/11/2013 11:09:59 have 2 jobs, 101 & 102 04/11/2013 11:10:00 – 04/11/2013 11:19:59 have 2 jobs, 101 & 102 04/11/2013 11:20:00 – 04/11/2013 11:29:59 have 3 jobs, 101 & 102 & 103 … 04/11/2013 11:50:00 – 04/11/2013 11:59:59 have 2 jobs, 101 & 105 04/11/2013 12:00:00 – 04/11/2013 12:09:59 have 0 job I wonder if you can give me a help……thanks a lot

  Read the article

• Limiting memory usage and mimimizing swap thrashing on Unix / Linux

  - by camelccc

  I have a few machines that I machine that I use for running large numbers of jobs where I try to limit the number of jobs so as not to exceed the available RAM of the machine. Occasionally I mis-estimate how much memory some of the jobs will take, and the machine starts thrashing the swap file. I resolve this by sending the kill -s STOP to one of the jobs so that it can get swapped out. Does anyone know of a utility that will monitor a server for processes by a specific name, and then pause the one with the smallest memory footprint is the total memory consumption reaches a desired threshold so that the larger ones can run and complete with a minimum of swap file thrashing? Paused processes then need to be resumed once some existing processes have completed.

  Read the article

• For those of you who are senior developers what do you look for in a new company and development team?

  - by Amy P

  As I move forward in my career new jobs become more difficult to choose between. When I was starting out and for the first 8 years of my career I took the jobs that I could get that would keep me programming on the general technological path that I was on. I am a job hopper, I only stay with a company for between 2 - 3 years. I think that I do this because after 2 years I get bored and unless there are new projects to keep my busy I no longer find work interesting. Now that I am becoming more experienced it is more important for me to only apply for jobs that are interesting and will move my career and my skill set forward. My problem now is that I keep finding jobs where the projects appear to be interesting during the interview but once I get in the company I find the development environment is sub-par and the development team is disjointed. I feel like I am asking the wrong questions during the interview process and don't know what to look for to make sure that the environment I will be working in will be a good one. Now my question: For those of you who are senior developers what do you look for in a new company and development team? I am looking for the key qualities in a company and development team that you look for when interviewing with a company. These qualities are the ones that would give you hints that the company will be a good one to work for.

  Read the article

• Renaming the sa Account

  - by Tara Kizer

  Today I renamed the sa account on 23 SQL Server 2005/2008 instances.  I used the CMS to assist with this task.  Later we realized all of the SQL Agent jobs were failing on these instances with the following error: “The job failed.  The owner (sa) of job XYZ does not have server access.” We use sa as the job owner, and it had correctly changed the owner to our new name as the sids had not changed.  We were at first confused why the jobs were failing but then realized restarting the SQL Agent service might help.  The restart corrected the problem. If you plan on renaming your sa account (best practice), make sure you restart the SQL Agent service afterwards to avoid failing jobs.  This is perhaps common knowledge, but it was something new learned by me today.

  Read the article

• How do I apply a computer science degree to web development?

  - by T. Webster

  I'm a web programmer, but I haven't found many opportunities to take advantage of a formal education in computer science. Maybe I'm not looking in the right places, but it seems to me like most of the web jobs I come across are CRUD, web forms, and data grids. For these jobs a formal CS background doesn't seem necessary, and you could do fine with O'Reilly cookbooks in jQuery, CSS 3, PHP, SQL, or ASP.NET MVC. What kinds of web developer jobs exist that really let you apply your computer science background? Do I need to branch out into other areas of programming to take full advantage of my degree?

  Read the article

• When designing a job queue, what should determine the scope of a job?

  - by Stuart Pegg

  We've got a job queue system that'll cheerfully process any kind of job given to it. We intend to use it to process jobs that each contain 2 tasks: Job (Pass information from one server to another) Fetch task (get the data, slowly) Send task (send the data, comparatively quickly) The difficulty we're having is that we don't know whether to break the tasks into separate jobs, or process the job in one go. Are there any best practices or useful references on this subject? Is there some obvious benefit to a method that we're missing? So far we can see these benefits for each method: Split Job lease length reflects job length: Rather than total of two Finer granularity on recovery: If we lose outgoing connectivity we can tell them all to retry The starting state of the second task is saved to job history: Helps with debugging (although similar logging could be added in single task method) Single Single job to be scheduled: Less processing overhead Data not stale on recovery: If the outgoing downtime is quite long, the pending Send jobs could be outdated

  Read the article

• Windows Azure HPC Scheduler Architecture

  - by Churianov Roman

  So far I've found very little information on the scheduling policy, resource management policy of Azure HPC Scheduler. I would appreciate any kind of information regarding some of these questions: What scheduling policy does a Head Node use to scatter jobs to Compute Nodes? Does Azure Scheduler use prior information about the jobs (compute time, memory demands ...) ? If 'yes', how it gets this information? Does Azure Scheduler split a job into several parallel jobs on one Compute node? Does it have any protection from Compute Node failures? (what it does when a compute node stops responding) Does it support addition/subtraction of Compute nodes? Is it possible to cancel a job? P.S. I'm aware of the MSDN resource Windows Azure HPC Scheduler. I found only information of how to use this Scheduler but almost nothing about how it works inside.

  Read the article

• seo for complex url

  - by Mark

  I have a jobs website. There are categories and sub categories e.g the url for builders is www.mydomain.com/jobs/all/active/202/tradesmen/799/builders/county/all Because of the framework im using (Elgg), the "all" and "active" come after the plugin name. Also, i need to do a lot of things with the url: get the pplugin (job), get the correct type of entities (all and active), get the category (202 and tradesmen), get the subcategory (799 & builders) and get the location (county and all). Is this URL seo friendly or is it too messy? Is something like www.mydomain.com/jobs/tradesmen/builders/all better?

  Read the article

• Cheap and Affordable SEO Services

  These days it is very easy to find cheap SEO services in India. It is because the unemployment is at its peak and people have started to move towards jobs from homes. These jobs from home include all types of online jobs but SEO has more demand than any other. This is an obvious thing that if you are selling any item and that item has a lot of competition then only the person with cheapest price along with the best quality will be successful. As SEO services are very common therefore you can find very cheap SEO person.

  Read the article

• Oracle Big Data Software Downloads

  - by Mike.Hallett(at)Oracle-BI&EPM

  Companies have been making business decisions for decades based on transactional data stored in relational databases. Beyond that critical data, is a potential treasure trove of less structured data: weblogs, social media, email, sensors, and photographs that can be mined for useful information. Oracle offers a broad integrated portfolio of products to help you acquire and organize these diverse data sources and analyze them alongside your existing data to find new insights and capitalize on hidden relationships. Oracle Big Data Connectors Downloads here, includes: Oracle SQL Connector for Hadoop Distributed File System Release 2.1.0 Oracle Loader for Hadoop Release 2.1.0 Oracle Data Integrator Companion 11g Oracle R Connector for Hadoop v 2.1 Oracle Big Data Documentation The Oracle Big Data solution offers an integrated portfolio of products to help you organize and analyze your diverse data sources alongside your existing data to find new insights and capitalize on hidden relationships. Oracle Big Data, Release 2.2.0 - E41604_01 zip (27.4 MB) Integrated Software and Big Data Connectors User's Guide HTML PDF Oracle Data Integrator (ODI) Application Adapter for Hadoop Apache Hadoop is designed to handle and process data that is typically from data sources that are non-relational and data volumes that are beyond what is handled by relational databases. Typical processing in Hadoop includes data validation and transformations that are programmed as MapReduce jobs. Designing and implementing a MapReduce job usually requires expert programming knowledge. However, when you use Oracle Data Integrator with the Application Adapter for Hadoop, you do not need to write MapReduce jobs. Oracle Data Integrator uses Hive and the Hive Query Language (HiveQL), a SQL-like language for implementing MapReduce jobs. Employing familiar and easy-to-use tools and pre-configured knowledge modules (KMs), the application adapter provides the following capabilities: Loading data into Hadoop from the local file system and HDFS Performing validation and transformation of data within Hadoop Loading processed data from Hadoop to an Oracle database for further processing and generating reports Oracle Database Loader for Hadoop Oracle Loader for Hadoop is an efficient and high-performance loader for fast movement of data from a Hadoop cluster into a table in an Oracle database. It pre-partitions the data if necessary and transforms it into a database-ready format. Oracle Loader for Hadoop is a Java MapReduce application that balances the data across reducers to help maximize performance. Oracle R Connector for Hadoop Oracle R Connector for Hadoop is a collection of R packages that provide: Interfaces to work with Hive tables, the Apache Hadoop compute infrastructure, the local R environment, and Oracle database tables Predictive analytic techniques, written in R or Java as Hadoop MapReduce jobs, that can be applied to data in HDFS files You install and load this package as you would any other R package. Using simple R functions, you can perform tasks such as: Access and transform HDFS data using a Hive-enabled transparency layer Use the R language for writing mappers and reducers Copy data between R memory, the local file system, HDFS, Hive, and Oracle databases Schedule R programs to execute as Hadoop MapReduce jobs and return the results to any of those locations Oracle SQL Connector for Hadoop Distributed File System Using Oracle SQL Connector for HDFS, you can use an Oracle Database to access and analyze data residing in Hadoop in these formats: Data Pump files in HDFS Delimited text files in HDFS Hive tables For other file formats, such as JSON files, you can stage the input in Hive tables before using Oracle SQL Connector for HDFS. Oracle SQL Connector for HDFS uses external tables to provide Oracle Database with read access to Hive tables, and to delimited text files and Data Pump files in HDFS. Related Documentation Cloudera's Distribution Including Apache Hadoop Library HTML Oracle R Enterprise HTML Oracle NoSQL Database HTML Recent Blog Posts Big Data Appliance vs. DIY Price Comparison Big Data: Architecture Overview Big Data: Achieve the Impossible in Real-Time Big Data: Vertical Behavioral Analytics Big Data: In-Memory MapReduce Flume and Hive for Log Analytics Building Workflows in Oozie

  Read the article

• Threading Overview

  - by ACShorten

  One of the major features of the batch framework is the ability to support multi-threading. The multi-threading support allows a site to increase throughput on an individual batch job by splitting the total workload across multiple individual threads. This means each thread has fine level control over a segment of the total data volume at any time. The idea behind the threading is based upon the notion that "many hands make light work". Each thread takes a segment of data in parallel and operates on that smaller set. The object identifier allocation algorithm built into the product randomly assigns keys to help ensure an even distribution of the numbers of records across the threads and to minimize resource and lock contention. The best way to visualize the concept of threading is to use a "pie" analogy. Imagine the total workset for a batch job is a "pie". If you split that pie into equal sized segments, each segment would represent an individual thread. The concept of threading has advantages and disadvantages: Smaller elapsed runtimes - Jobs that are multi-threaded finish earlier than jobs that are single threaded. With smaller amounts of work to do, jobs with threading will finish earlier. Note: The elapsed runtime of the threads is rarely proportional to the number of threads executed. Even though contention is minimized, some contention does exist for resources which can adversely affect runtime. Threads can be managed individually – Each thread can be started individually and can also be restarted individually in case of failure. If you need to rerun thread X then that is the only thread that needs to be resubmitted. Threading can be somewhat dynamic – The number of threads that are run on any instance can be varied as the thread number and thread limit are parameters passed to the job at runtime. They can also be configured using the configuration files outlined in this document and the relevant manuals.Note: Threading is not dynamic after the job has been submitted Failure risk due to data issues with threading is reduced – As mentioned earlier individual threads can be restarted in case of failure. This limits the risk to the total job if there is a data issue with a particular thread or a group of threads. Number of threads is not infinite – As with any resource there is a theoretical limit. While the thread limit can be up to 1000 threads, the number of threads you can physically execute will be limited by the CPU and IO resources available to the job at execution time. Theoretically with the objects identifiers evenly spread across the threads the elapsed runtime for the threads should all be the same. In other words, when executing in multiple threads theoretically all the threads should finish at the same time. Whilst this is possible, it is also possible that individual threads may take longer than other threads for the following reasons: Workloads within the threads are not always the same - Whilst each thread is operating on the roughly the same amounts of objects, the amount of processing for each object is not always the same. For example, an account may have a more complex rate which requires more processing or a meter has a complex amount of configuration to process. If a thread has a higher proportion of objects with complex processing it will take longer than a thread with simple processing. The amount of processing is dependent on the configuration of the individual data for the job. Data may be skewed – Even though the object identifier generation algorithm attempts to spread the object identifiers across threads there are some jobs that use additional factors to select records for processing. If any of those factors exhibit any data skew then certain threads may finish later. For example, if more accounts are allocated to a particular part of a schedule then threads in that schedule may finish later than other threads executed. Threading is important to the success of individual jobs. For more guidelines and techniques for optimizing threading refer to Multi-Threading Guidelines in the Batch Best Practices for Oracle Utilities Application Framework based products (Doc Id: 836362.1) whitepaper available from My Oracle Support

  Read the article

• iphone xcode - different levels of drill down in tab bar controller - nav controller - table view

  - by Frames84

  I have this type of data categories: today news - news item jobs news - news item general news - sub category news - news item So i have followed the very good tutorial 'Building an iPhone App Combining Tab Bar, Navigation and Tab' http://www.youtube.com/watch?v=LBnPfAtswgw and all is good with the 1st two 'todays news' and 'jobs news' but can't figure out the best method for implementing the sub category table view? Do i use the same table view but reload it with the sub categories then some how work out when one is click that it's a sub category or is there a better method? This is how i'm set up in Main Window.Xib - Tab Bar Controller - -Navigation Controller (today) - - - Table View (list of today news) - -Navigation Controller (jobs) - - - Table View (list of jobs news) - -Navigation Controller (general) - - - Table View (general sub cat) // how to implement this Table View - - - - Table View (list of sub cat news) Thanks for your time

  Read the article

• Get-WMIObject fails when run AsJob

  - by codepoke

  It's a simple question, but it's stumped me. $cred = Get-Credential $jobs = @() $jobs += Get-WmiObject ` -Authentication 6 ` -ComputerName 'serverName' ` -Query 'Select * From IISWebServerSetting' ` -Namespace 'root/microsoftiisv2' ` -EnableAllPrivileges ` -Credential $cred ` -Impersonation 4 ` -AsJob $joblist = Wait-Job -Job $jobs -Timeout 60 foreach ($job in $jobs) { if ($job.State -eq "Completed") { $app = Receive-Job -Job $job $app } else { ("Job not completed: " + $job.Name + "@" + $job.State + ". Reason:" + $job.ChildJobs[0].JobStateInfo.Reason) Remove-Job -Job $job -Force } } The query succeeds when run directly and fails when run -AsJob. Reason:System.UnauthorizedAccessException: Access is denied. I've jiggered with -Impersonation, -Credentials, -Authority, and -EnableAllPrivileges to no useful effect. It appears I'm overlooking something fundamental. Why is my Powershell prompt allowed to connect to the remote server, but my child process denied?

  Read the article

• Using will_paginate with AJAX live search with jQuery in Rails

  - by Mark Richman

  I am using will_paginate to successfully page through records. I am also using AJAX live search via jQuery to update my results div. No problem so far. The issue I have is when trying to paginate through those live search results. I simply get "Page is loading..." with no div update. Am I missing something fundamental? # index.html.erb <form id="searchform" accept-charset="utf-8" method="get" action="/search"> Search: <input id="search" name="search" type="text" autocomplete="off" title="Search location, company, description..." /> <%= image_tag("spinner.gif", :id => "spinner", :style =>"display: none;" ) %> </form> # JobsController#search def search if params[:search].nil? @jobs = Job.paginate :page => params[:page], :order => "created_at desc" elsif params[:search] and request.xhr? @jobs = Job.search params[:search], params[:page] end render :partial => "jobs", :layout => false, :locals => { :jobs => @jobs } end # Job#search def self.search(search, page) logger.debug "Job.paginate #{search}, #{page}" paginate :per_page => @@per_page, :page => page, :conditions => ["description LIKE ? or title LIKE ? or company LIKE ?", "%#{search}%", "%#{search}%", "%#{search}%"], :order => 'created_at DESC' end # search.js $(document).ready(function(){ $("#search").keyup(function() { $("#spinner").show(); // show the spinner var form = $(this).parents("form"); // grab the form wrapping the search bar. var url = form.attr("action"); // grab the URL from the form's action value. var formData = form.serialize(); // grab the data in the form $.get(url, formData, function(html) { // perform an AJAX get, the trailing function is what happens on successful get. $("#spinner").hide(); // hide the spinner $("#jobs").html(html); // replace the "results" div with the result of action taken }); }); });

  Read the article

• Why does Celery work in Python shell, but not in my Django views? (import problem)

  - by TIMEX

  I installed Celery (latest stable version.) I have a directory called /home/myuser/fable/jobs. Inside this directory, I have a file called tasks.py: from celery.decorators import task from celery.task import Task class Submitter(Task): def run(self, post, **kwargs): return "Yes, it works!!!!!!" Inside this directory, I also have a file called celeryconfig.py: BROKER_HOST = "localhost" BROKER_PORT = 5672 BROKER_USER = "abc" BROKER_PASSWORD = "xyz" BROKER_VHOST = "fablemq" CELERY_RESULT_BACKEND = "amqp" CELERY_IMPORTS = ("tasks", ) In my /etc/profile, I have these set as my PYTHONPATH: PYTHONPATH=/home/myuser/fable:/home/myuser/fable/jobs So I run my Celery worker using the console ($ celeryd --loglevel=INFO), and I try it out. I open the Python console and import the tasks. Then, I run the Submitter. >>> import fable.jobs.tasks as tasks >>> s = tasks.Submitter() >>> s.delay("abc") <AsyncResult: d70d9732-fb07-4cca-82be-d7912124a987> Everything works, as you can see in my console [2011-01-09 17:30:05,766: INFO/MainProcess] Task tasks.Submitter[d70d9732-fb07-4cca-82be-d7912124a987] succeeded in 0.0398268699646s: But when I go into my Django's views.py and run the exact 3 lines of code as above, I get this: [2011-01-09 17:25:20,298: ERROR/MainProcess] Unknown task ignored: "Task of kind 'fable.jobs.tasks.Submitter' is not registered, please make sure it's imported.": {'retries': 0, 'task': 'fable.jobs.tasks.Submitter', 'args': ('abc',), 'expires': None, 'eta': None, 'kwargs': {}, 'id': 'eb5c65b4-f352-45c6-96f1-05d3a5329d53'} Traceback (most recent call last): File "/home/myuser/mysite-env/lib/python2.6/site-packages/celery/worker/listener.py", line 321, in receive_message eventer=self.event_dispatcher) File "/home/myuser/mysite-env/lib/python2.6/site-packages/celery/worker/job.py", line 299, in from_message eta=eta, expires=expires) File "/home/myuser/mysite-env/lib/python2.6/site-packages/celery/worker/job.py", line 243, in __init__ self.task = tasks[self.task_name] File "/home/myuser/mysite-env/lib/python2.6/site-packages/celery/registry.py", line 63, in __getitem__ raise self.NotRegistered(str(exc)) NotRegistered: "Task of kind 'fable.jobs.tasks.Submitter' is not registered, please make sure it's imported." It's weird, because the celeryd client does show that it's registered, when I launch it. [2011-01-09 17:38:27,446: WARNING/MainProcess] Configuration -> . broker -> amqp://GOGOme@localhost:5672/fablemq . queues -> . celery -> exchange:celery (direct) binding:celery . concurrency -> 1 . loader -> celery.loaders.default.Loader . logfile -> [stderr]@INFO . events -> OFF . beat -> OFF . tasks -> . tasks.Decayer . tasks.Submitter Can someone help?

  Read the article

• Tuple struct constructor complains about private fields

  - by Grubermensch

  I am working on a basic shell interpreter to familiarize myself with Rust. While working on the table for storing suspended jobs in the shell, I have gotten stuck at the following compiler error message: tsh.rs:8:18: 8:31 error: cannot invoke tuple struct constructor with private fields tsh.rs:8 let mut jobs = job::JobsList(vec![]); ^~~~~~~~~~~~~ It's unclear to me what is being seen as private here. As you can see below, both of the structs are tagged with pub in my module file. So, what's the secret sauce? tsh.rs use std::io; mod job; fn main() { // Initialize jobs list let mut jobs = job::JobsList(vec![]); loop { /*** Shell runtime loop ***/ } } job.rs use std::fmt; pub struct Job { jid: int, pid: int, cmd: String } impl fmt::Show for Job { /*** Formatter ***/ } pub struct JobsList(Vec<Job>); impl fmt::Show for JobsList { /*** Formatter ***/ }

  Read the article

< Previous Page | 17 18 19 20 21 22 23 24 25 26 27 28  | Next Page >