Search Results

Search found 42919 results on 1717 pages for 'web crawler'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 453/1717 | < Previous Page | 449 450 451 452 453 454 455 456 457 458 459 460  | Next Page >

  • rcurl web scraping timeout exits program

    - by user1742368
    I am using a loop and rcurl scrape data from multiple pages which seems to work fine at certain times but fails when there is a timeout due to the server not responding. I am using a timeout=30 which traps the timeout error however the program stops after the timeout. i would like the progrm to continue to the next page when the timeout occurrs but cant figureout how to do this? url = getCurlHandle(cookiefile = "", verbose = TRUE) Here is the statement I am using that causes the timeout. I am happy to share the code if there is interest. webpage = getURLContent(url, followlocation=TRUE, curl = curl,.opts=list( verbose = TRUE, timeout=90, maxredirs = 2)) woodwardjj

    Read the article

  • How is the "click to view more" function implemented?

    - by bobo
    We often see websites that display first few lines of an article and then append ... [More] so that people who are interested in can click on it to view the full article. To implement this functionality, we first need to find out where the article text should be cut to append the ... [More]. Since there must be some HTML/ CSS coupled with the article text, we must remember to ignore them when calculating the length of the text. But there is something I just can't figure it out, if more than one language is mixed in the first few lines of the article text, the length of the text will be very difficult to calculate since the characters can be of variable length. How should we solve this problem?

    Read the article

  • jKey (JavaScript key shortcut plugin) Issue

    - by Oscar Godson
    Me and a friend are writing a plugin for jQuery that makes it easy for devs to add key shortcuts and we're damn close but no cigar. We're having issues with the key combos. It seems like we are having issues when you call the same selector multiple times on a page. Try pressing alt+a... youll see it works one time, then gets all mangled up. Anyone know how to fix it? It'll be on github after it's corrected and I'd be happy to add "thank you to" link to whoever can fix this in the header with the copyright info :) It's nicely documented and i have all the code and stuff here. So... anyone? http://jsbin.com/azaha4

    Read the article

  • Android UnknownHost in asyncTask - loading web page

    - by Sneha
    I followed this tutorial for AsyncTask and getting the following error log: 03-23 11:44:42.936: WARN/System.err(315): java.net.UnknownHostException: www.google.co.in 03-23 11:44:42.936: WARN/System.err(315): at java.net.InetAddress.lookupHostByName(InetAddress.java:513) 03-23 11:44:42.936: WARN/System.err(315): at java.net.InetAddress.getAllByNameImpl(InetAddress.java:278) 03-23 11:44:42.936: WARN/System.err(315): at java.net.InetAddress.getAllByName(InetAddress.java:242) 03-23 11:44:42.936: WARN/System.err(315): at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:136) 03-23 11:44:42.936: WARN/System.err(315): at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:164) 03-23 11:44:42.936: WARN/System.err(315): at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:119) 03-23 11:44:42.936: WARN/System.err(315): at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:348) 03-23 11:44:42.936: WARN/System.err(315): at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:555) 03-23 11:44:42.936: WARN/System.err(315): at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:487) 03-23 11:44:42.944: WARN/System.err(315): at org.apache.http.impl.client.AbstractHttpC lient.execute(AbstractHttpClient.java:465) 03-23 11:44:42.944: WARN/System.err(315): at com.test.async.AsyncTaskExampleActivity$DownloadWebPageTask.doInBackground (AsyncTaskExampleActivity.java:36) 03-23 11:44:42.944: WARN/System.err(315): at com.test.async.AsyncTaskExampleActivity$DownloadWebPageTask.doInBackground (AsyncTaskExampleActivity.java:1) 03-23 11:44:42.944: WARN/System.err(315): at android.os.AsyncTask$2.call(AsyncTask.java:185) 03-23 11:44:42.944: WARN/System.err(315): at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305) 03-23 11:44:42.944: WARN/System.err(315): at java.util.concurrent.FutureTask.run (FutureTask.java:137) 03-23 11:44:42.944: WARN/System.err(315): at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1068) 03-23 11:44:42.944: WARN/System.err(315): at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:561) 03-23 11:44:42.944: WARN/System.err(315): at java.lang.Thread.run(Thread.java:1096) How do i fix it?? My Code: public class AsyncTaskExampleActivity extends Activity { private TextView textView; /** Called when the activity is first created. */ @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.main); textView = (TextView) findViewById(R.id.TextView01); } private class DownloadWebPageTask extends AsyncTask<String, Void, String> { @Override protected String doInBackground(String... urls) { String response = ""; Log.i("", "in doInBackgroundddddddddd.........."); Log.i("", "in readWebpageeeeeeeeeeeee"); /* * try { InetAddress i = * InetAddress.getByName("http://google.co.in"); } catch * (UnknownHostException e1) { e1.printStackTrace(); } */ for (String url : urls) { Log.i("", "in for looooooop doInBackgroundddddddddd.........."); DefaultHttpClient client = new DefaultHttpClient(); HttpGet httpGet = new HttpGet(url); try { Log .i("", "afetr for looooooop try doInBackgroundddddddddd.........."); HttpResponse execute = client.execute(httpGet); Log .i("", "afetr for looooooop try client ..execute doInBackgroundddddddddd.........."); InputStream content = execute.getEntity().getContent(); BufferedReader buffer = new BufferedReader( new InputStreamReader(content)); String s = ""; while ((s = buffer.readLine()) != null) { response += s; Log .i("", "afetr while looooooop try client ..execute doInBackgroundddddddddd.........."); } } catch (Exception e) { e.printStackTrace(); } } Log .i("", "afetr lasttttttttttttt b4 response doInBackgroundddddddddd.........."); return response; } @Override protected void onPostExecute(String result) { Log.i("", "in onPostExecuteeee.........."); textView.setText(result); } } public void readWebpage(View view) { /* * System.setProperty("http.proxyHost", "10.132.116.10"); * System.setProperty("http.proxyPort", "3128"); */ DownloadWebPageTask task = new DownloadWebPageTask(); task.execute(new String[] { "http://google.co.in" }); Log.i("", "in readWebpageeeeeeeeeeeee after execute.........."); } } main.xml: <Button android:id="@+id/readWebpage" android:layout_width="match_parent" android:layout_height="wrap_content" android:onClick="readWebpage" android:text="Load Webpage"> </Button> <TextView android:id="@+id/TextView01" android:layout_width="match_parent" android:layout_height="match_parent" android:text="Example Text"> </TextView> Manifest: <?xml version="1.0" encoding="utf-8"?> <manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.test.async" android:versionCode="1" android:versionName="1.0"> <uses-sdk android:minSdkVersion="8" /> <uses-permission android:name="android.permission.INTERNET"></uses-permission> <application android:icon="@drawable/icon" android:label="@string/app_name"> <activity android:name=".AsyncTaskExampleActivity" android:label="@string/app_name"> <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LAUNCHER" /> </intent-filter> </activity> </application> Thanks Sneha

    Read the article

  • Sax parsing from web service

    - by donald
    Hey, I am trying to parse xml file using Sax parser. let's say xml is like this.. I want to count the number of times b element is present (its variable) And i want this count before parsing, so that I can declare an array of appropriate size. One way is to run count then separately and other way is dynamic array (List Array) Is there any other better way to do this? Also, Is it possible to make an ArrayList of my class..? because I want an array of type myClass.

    Read the article

  • Open source social network

    - by user72185
    I looked through the existing questions of this kind but didn't find what I was looking for so here goes. I need an open source framework where people can... Create user accounts Invite friends Have a friends list Write messages to each other (some kind of inbox system) Create posts on a standard message forum On top of this I will create some kind of game where users interact, so it must be posibble to extend the framework with new database tables and functionality. The platform is Windows, but other than that I'm willing to try pretty much anything. Thanks for suggestions!

    Read the article

  • Issues in Ajax based applications

    - by Sinuhe
    I'm very interested in developing Ajax based applications. This is, loading almost all of the content of the application via XMLHttpRequest, instead of only some combos and widgets. But if I try to do this form scratch, soon I find some problems without an easy solution. I wonder if there is some framework (both client and server side) to deal with this issues. As far as I know, there isn't (but I've searched mainly in Java world). So I am seriously thinking of doing my own framework, at least for my projects. Therefore, in this question I ask for several things. First, the possible problems of an ajax based development. Then, I'm looking for some framework or utility in order to deal with them. Finally, if there is no framework available, what features must it have. Here are the issues I thought: 1 - JavaScript must be enabled. Security paranoia isn't the only problem: a lot of mobile devices couldn't use the application, too. 2 - Sometimes you need to update more than one DIV (e.g. main content, menu and breadcrumbs). 3 - Unknown response type: when you make an Ajax call, you set the callback function too, usually specifying if expected response is a javascript object or in which DIV put the result. But this fails when you get another type of response: for example when the session has expired and the user must log in again. 4 - Browser's refresh, back and forward buttons can be a real pain. User will expect different behaviors depending on the situation. 5 - When search engines indexes a site, only follow links. Thus, content load by Ajax won't "exist" for who doesn't know about it yet. 6 - Users can ask for open a link in a different window/tab. 7 - Address bar doesn't show the "real" page you are in. So, you can't copy the location and send it to a friend or bookmark the page. 8 - If you want to monetize the site, you can put some advertisings. As you don't refresh entire page and you want to change the ad after some time, you have to refresh only the DIV where the ad is. But this can violate the Terms and Conditions of your ad service. In fact, it can go against AdSense TOS. 9 - When you refresh an entire page, all JavaScript gets "cleaned". But in Ajax calls, all JavaScript objects will remain. 10 - You can't easily change your CSS properties.

    Read the article

  • Web Form Security

    - by brandon14_99
    I set a cookie in the headers with an md5 hashed keyword. Then in my code, it checks for the exact matching cookie before displaying a form. Is this pretty much pointless? The form submits to an external site, so I am trying to secure the form without using captcha..

    Read the article

  • Can StructureMap be configured so that one can use different .config settings based on whether the p

    - by Mark Rogers
    I know that in StructureMap I can read from my *.config files (or files referenced by them), when I want to pass specific arguments to an object's constructor. ForRequestedType<IConfiguration>() .TheDefault.Is.OfConcreteType<SqlServerConfiguration>() .WithCtorArg("db_server_address") .EqualToAppSetting("data.db_server_address") But what I would like to do is read from one config setting in debug mode and another in release mode. Sure I could surround the .EqualToAppSetting("data.db_server_address"), with #if DEBUG, but for some reason those statements make me cringe a little when I put them in. I'd like to know if there was some way to do this with the StructureMap library itself. So can I feed my objects different settings based on whether the project is built in debug or release mode?

    Read the article

  • How to automate testing of a browser-based app?

    - by mawg
    If it were a windows program, I would use Auto it to automate testing. Is there something similar for browser-based apps? Nothing too complex, it should just allow scripting (preferable for me to macro-recording) to simulate human interaction with the browser, which means being able to identify fields of a form by name, inject text into some, simulate mouse-click on others, etc and then, after submitting a form, should be able to read text certain named controls, check the status of others (checked, radio group index, read-only, etc). While I do appreciate a full featured product, I don't appreciate a steep learning curve. so something as simple as the scripting of Auto It woudl be fine. I don't know if it makes a difference which browser is used, but I could live with MSIE 6 or higher (maybe 7 or higher at a push).

    Read the article

  • Debugging a Browser Redirect Loop

    - by just_wes
    Hi all, I am using CakePHP with the Auth and ACL components. My page loads fine for non-registered users, but if I try to log in as a registered user I get an infinite redirect loop in the browser. I am sure that this is some sort of permissions problem, but the problem exists even for users who have permissions for everything. The only way to prevent this behavior is to allow '*' in my AppController's beforeFilter method. What is the best way to debug this sort of problem? Thanks!

    Read the article

  • How to implement session like stackoverflow?

    - by stacker
    I'm implementing this kink of login: http://jaspan.com/improved_persistent_login_cookie_best_practice In this design a new token issued to the user each new login. So it tells me that I need to Now I need to implement a session, for this login. I'd like to implement session like stackoverflow, so people will can have session without login. but for a login there will be always a session. Any ideas how?

    Read the article

  • small scale web site - global javascript file style/format/pattern - improving maintainability

    - by yaya3
    I frequently create (and inherit) small to medium websites where I have the following sort of code in a single file (normally named global.js or application.js or projectname.js). If functions get big, I normally put them in a seperate file, and call them at the bottom of the file below in the $(document).ready() section. If I have a few functions that are unique to certain pages, I normally have another switch statement for the body class inside the $(document).ready() section. How could I restructure this code to make it more maintainable? Note: I am less interested in the functions innards, more so the structure, and how different types of functions should be dealt with. I've also posted the code here - http://pastie.org/999932 in case it makes it any easier var ProjectNameEnvironment = {}; function someFunctionUniqueToTheHomepageNotWorthMakingConfigurable () { $('.foo').hide(); $('.bar').click(function(){ $('.foo').show(); }); } function functionThatIsWorthMakingConfigurable(config) { var foo = config.foo || 700; var bar = 200; return foo * bar; } function globallyRequiredJqueryPluginTrigger (tooltip_string) { var tooltipTrigger = $(tooltip_string); tooltipTrigger.tooltip({ showURL: false ... }); } function minorUtilityOneLiner (selector) { $(selector).find('li:even').not('li ul li').addClass('even'); } var Lightbox = {}; Lightbox.setup = function(){ $('li#foo a').attr('href','#alpha'); $('li#bar a').attr('href','#beta'); } Lightbox.init = function (config){ if (typeof $.fn.fancybox =='function') { Lightbox.setup(); var fade_in_speed = config.fade_in_speed || 1000; var frame_height = config.frame_height || 1700; $(config.selector).fancybox({ frameHeight : frame_height, callbackOnShow: function() { var content_to_load = config.content_to_load; ... }, callbackOnClose : function(){ $('body').height($('body').height()); } }); } else { if (ProjectNameEnvironment.debug) { alert('the fancybox plugin has not been loaded'); } } } // ---------- order of execution ----------- $(document).ready(function () { urls = urlConfig(); (function globalFunctions() { $('.tooltip-trigger').each(function(){ globallyRequiredJqueryPluginTrigger(this); }); minorUtilityOneLiner('ul.foo') Lightbox.init({ selector : 'a#a-lightbox-trigger-js', ... }); Lightbox.init({ selector : 'a#another-lightbox-trigger-js', ... }); })(); if ( $('body').attr('id') == 'home-page' ) { (function homeFunctions() { someFunctionUniqueToTheHomepageNotWorthMakingConfigurable (); })(); } });

    Read the article

  • What percent of web sites use JavaScript?

    - by Claudiu
    I'm wondering just how pervasive JavaScript is. This article states that 73% of websites they tested rely on JavaScript for important functionality, but it seems to me that the number must be larger. Have any surveys been done on this topic? Maybe a better way to phrase this question is - are there any sites that don't use JavaScript? EDIT: By 'use', I don't necessarily mean "rely on for important functionality" - that was just the statistic that one article gave.

    Read the article

  • mootools accordion styling problem

    - by Midhat
    I just built my first mootools accordion, but it is adding a lot of inline styles which is just ruining my UI. I can set up a inline style with !important keyword but it will just make my css maintenance a nightmare. any ideas how to get rid of the inline styles

    Read the article

  • PHP Session variable isset(..)=1 after session_start()

    - by Nicsoft
    Hello! I guess I am not understanding the scope of session variables, or the session itself, in PHP, hence this question: This is my code if(!session_id()==""){ echo "Getting rid of session"."</br>"; session_destroy(); } echo "Before session_start(): ".isset($_SESSION["first_date_of_week"])."</br>"; session_start(); echo "After session_start(): ".isset($_SESSION["first_date_of_week"])." ".$_SESSION["first_date_of_week"]->format("Y-m-d")."</br>"; The output is: Before session_start(): After session_start(): 1 2011-01-09 How come that when doing the isset(..) on the session variable it is set directly after starting the session, even though I haven't even used it or set it yet? It does, however, still have the same value as before. Also, session_id()="" since the if-clause is never triggered. I never kill the session, how come it is set to ""? I.e. I refresh the page and expects the session to still be alive. Using the isset(..) function is then pretty useless testing if it has been set already... Thanks in advance! /Niklas

    Read the article

  • JavaScript/HTML: How do I display an IMG with a set dimension and if the image is wider or taller th

    - by NickNick
    I have a bunch of images that are guaranteed to have: minimum width = 200px maximum width = 250px minimum height = 150px maximum height = 175px What I want to do is display a consist 200px by 150px rectangle of the image while maintaining scale (no stretching or shrinking). Which means, I might have some overflow. How can I display the image so that it keeps porpotions to the original image size, yet displayed inside a 200x150 px window and hiding any overflow?

    Read the article

  • How Does WordPress Block Search Engines?

    - by Sarfraz
    Hello, If you go to wordpress admin and then settings-privacy, there are two options asking you whether you want to allow your blog to be searched though by seach engines and this option: I would like to block search engines, but allow normal visitors How does wordpress actually block search bots/crawlers from searching through this site when the site is live?

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 449 450 451 452 453 454 455 456 457 458 459 460  | Next Page >