Search Results

Search found 10391 results on 416 pages for 'sys dm exec requests'.

Page 136/416 | < Previous Page | 132 133 134 135 136 137 138 139 140 141 142 143  | Next Page >

  • How to work around a site forbidding me to scrape their images with PHP

    - by Petruza
    I'm scraping a site, searching for JPGs to download. Scraping the site's HTML pages works fine. But when I try getting the JPGs with CURL, copy(), fopen(), etc., I get a 403 forbiden status. I know that's because the site owners don't want their images scraped, so I understand a good answer would be just don't do it, because they don't want you to. Ok, but let's say it's ok and I try to work around this, how could this be achieved? If I get the same URL with a browser, I can open the image perfectly, it's not that my IP is banned or anything, and I'm testing the scraper one file at a time, so it's not blocking me because I make too many requests too often. From my understanding, it could be that either the site is checking for some cookies that confirm that I'm using a browser and browsing their site before I download a JPG. Or that maybe PHP is using some user agent for the requests that the server can detect and filter out. Anyway, have any idea?

    Read the article

  • Android: Maps - Best way to provide 'search for location' feature?

    - by r3mo
    Hello all, I've got an android app that uses a map activity and serves up content based on map location. I'm looking for a way to allow the user to search for a location by name (anything from 'New York' to 'Eiffel Tower') - e.g. have a text input field into which they could type 'Rome' - after pressing a button, the user would be brought to the coordinates of Rome on the map. What would be the best way to go about this? I've looked into the google geocoding api (http://code.google.com/apis/maps/documentation/geocoding/), but it has limitations of 2,500 geolocation requests per day - I'm presuming this is per API key? Or is it per user/source IP? 2,500 requests for one android app woudln't last long. Ideally, I would be able to search for English and foreign names of countries. Thanks in advance! r3mo

    Read the article

  • Is it bad practice to select upstream servers based upon the HTTP method?

    - by PartlyCloudy
    I'm wondering if it is bad practice to have a reverse proxy that selects the upstream server depending on the HTTP method used? The background is that I have an abitrary web server that handles POST requests with some logic behind. The same resources also contain static content, that can be retrieved using GET. After some benchmarking I realized that nginx would handle the static content way faster than my abitrary web server doing this. I checked the option to forward incoming requests internally using nginx, which is feasible. But this would lead to the fact that different servers would serve a distinct resource, only depending on issuing a GET or POST, including different header fields.

    Read the article

  • Performance optimization for mssql: decrease stored procedures execution time or unload the server?

    - by tim
    Hello everybody! We have a web service which provides search over hotels. There is a problem with performance: a single request to the service takes around 5000 ms. Almost all of the time is spent in database by executing storing procedures. During the request our server (mssql2008) consumes ~90% of the processor time. When 2 requests are made in parallel the average time grows and is around 7000 ms. When number of request is increasing, the average time of response is increasing as well. We have 20-30 requests per minute. Which kind of optimization is the best in this case having in mind that the goal is to provide stable response time for the service: 1) Try to decrease the stored procedures execution time 2) Try to find the way how to unload the server It is interesting to hear from people who deal with booking sites. Thanks!

    Read the article

  • Can I learn the iPhone SDK and complete an app by September?

    - by Kuro
    Hi everyone, I'm wondering if it's realistically possible to learn the iPhone SDK and complete an app by the end of this summer. The app shouldn't be too difficult, it would just interact with my site's API and essentially just fetch, create, and edit resources. I already have a little experience with the SDK (and I'm comfortable with Objective-C), so I'm not completely new to it. Although here I things I don't know how to do and what I'd need to learn: Log in to my site using basic http authentication and store the user's username/password for all future requests Implement a toolbar at the bottom to navigate between different views Fetch and parse xml into list items Do POST/PUT requests to create/edit a resource on my server Are there any good resources available to learn how to do those tasks above? Thanks!

    Read the article

  • Multible jquery .load() methods not running concurrently in MVC 2

    - by Boob
    In one of my aspx pages i have something like this in the head: <script type="text/javascript"> $(function() { // Jquery stuff loadStuff(); }); function loadStuff() { $('#result1').load({ source: url }); $('#result2').load({ source: url }); $('#result3').load({ source: url }); }; </script> The idea being that as the page is being displayed these loads will populate the specified divs with results from different webservices as and when the info becomes available. My problem is that this requests are being queued and being sent one at a time. However I want these to be sent at the same time and whichever returns first gets displayed first! Also if I try to kick off another request while these divs are loading the request seems to get queued after the others. How do I make these requests concurrent instead of queued? Thanks

    Read the article

  • Python lxml - returns null list

    - by Chris Finlayson
    I cannot figure out what is wrong with the XPATH when trying to extract a value from a webpage table. The method seems correct as I can extract the page title and other attributes, but I cannot extract the third value, it always returns an empty list? from lxml import html import requests test_url = 'SC312226' page = ('https://www.opencompany.co.uk/company/'+test_url) print 'Now searching URL: '+page data = requests.get(page) tree = html.fromstring(data.text) print tree.xpath('//title/text()') # Get page title print tree.xpath('//a/@href') # Get href attribute of all links print tree.xpath('//*[@id="financial"]/table/tbody/tr/td[1]/table/tbody/tr[2]/td[1]/div[2]/text()') Unless i'm missing something, it would appear the XPATH is correct: Chrome screenshot I checked Chrome console, appears ok! So i'm at a loss $x ('//*[@id="financial"]/table/tbody/tr/td[1]/table/tbody/tr[2]/td[1]/div[2]/text()') [ "£432,272" ]

    Read the article

  • Authenticating model - best practices

    - by zerkms
    I come into ASP.NET from php so the reason why i ask my question is because it's totally different nature of how application works and handles requests. well, i have an exists table with user creditians, such as: id, login, password (sha hashed), email, phone, room i have built custom membership provider so it can handle my own database authentication schema. and now i'm confused, because User.Identity.Name contains only user's login, but not the complete object (i'm using linq2sql to communicate with database and i need in it's User object to work). at php applications i just store user object at some static method at Auth class (or some another), but here at ASP.NET MVC i cannot do this, because static member is shared across all requests and permanent, and not lives within only current request (as it was at php). so my question is: how and where should i retrieve and store linq2sql user object to work with it within current and only current request? (after request processed successfully i expect it will be disposed from memory and on next request will be created again). or i'm following totally wrong way?

    Read the article

  • Render view as string when Ajax request to view is made

    - by mare
    I have my views render to route requests like /controller/action (for instance, /Page/Create), so pretty classic. I am creating a form where view will be loaded through AJAX request with jQuery get(), so essentially I need my Views or to be precise, my controller actions, to return plain HTML string when an AJAX request is made to the route that renders the View. I want to use the same views as for normal requests, so I cannot afford to create any new views. It is something like calling RenderPartial("MyViewUserControl") but from inside the JS code.

    Read the article

  • No supported translation to SQL

    - by derans
    We have this code: private IList<InfoRequest> GetBy(Func<InformationRequest, string> func, string searchby) { var requests = _dc.InformationRequests .Where(x => func.Invoke(x).Contains(searchby)) .OrderBy(y => y.RequestDate); return Mapper.Map<InformationRequest[], InfoRequest[]>(requests.ToArray()); } It continues to throw the no supported translation to SQL error. Any ideas on the problem or how to resolve it?

    Read the article

  • Is there any way around the Chrome 5 breakage of Ajax for local files?

    - by nikow
    The recent Chrome 5.0 release completely blocks XMLHTTPRequest requests for local files comming from a local file. Here is just one of the many related bug reports and here is the code change (there is also a SO question caused by this). This breaks a lot of production code, e.g. for documentation systems. Users must be able to browse local documentation without the need to install anything or run executables. My question is if there is any way around this restriction? I'm only interested in solutions that don't require any fancy actions on the users side (nothing beyond a confirmation dialog). Is there any way the HTML5 File API could be used, or maybe postMessage()? Of course this whole issue is very frustrating for many people. Firefox takes a fare more reasonable approach and allows requests inside the directory. So it seems unlikely that other browser vendors will follow Chrome.

    Read the article

  • Clarifying... So Background Jobs don't Tie Up Application Resources (in Rails)?

    - by viatropos
    I'm trying to get a better grasp of the inner workings of background jobs and how they improve performance. I understand that the goal is to have the application return a response to the user as fast as it can, so you don't want to, say, parse a huge feed that would take 10 seconds because it would prevent the application from being able to process any other requests. So it's recommended to put any operations that take more than say 500ms to execute, into a queued background job. What I don't understand is, doesn't that just delay the same problem? I know the user who invoked that background job will get an immediate response, but what if another user comes right when that background job starts (and it takes 10 seconds to finish), wont that user have to wait? Or is the main issue that, requests are the only thing that can happen one-at-a-time, while on the other hand a request can start while one+ background jobs are in the middle of running? Is that correct?

    Read the article

  • Radius Authorization against ActiveDirectory and the users file

    - by mohrphium
    I have a problem with my freeradius server configuration. I want to be able to authenticate users against Windows ActiveDirectory (2008 R2) and the users file, because some of my co-workers are not listed in AD. We use the freeradius server to authenticate WLAN users. (PEAP/MSCHAPv2) AD Authentication works great, but I still have problems with the /etc/freeradius/users file When I run freeradius -X -x I get the following: Mon Jul 2 09:15:58 2012 : Info: ++++[chap] returns noop Mon Jul 2 09:15:58 2012 : Info: ++++[mschap] returns noop Mon Jul 2 09:15:58 2012 : Info: [suffix] No '@' in User-Name = "testtest", looking up realm NULL Mon Jul 2 09:15:58 2012 : Info: [suffix] Found realm "NULL" Mon Jul 2 09:15:58 2012 : Info: [suffix] Adding Stripped-User-Name = "testtest" Mon Jul 2 09:15:58 2012 : Info: [suffix] Adding Realm = "NULL" Mon Jul 2 09:15:58 2012 : Info: [suffix] Authentication realm is LOCAL. Mon Jul 2 09:15:58 2012 : Info: ++++[suffix] returns ok Mon Jul 2 09:15:58 2012 : Info: [eap] EAP packet type response id 1 length 13 Mon Jul 2 09:15:58 2012 : Info: [eap] No EAP Start, assuming it's an on-going EAP conversation Mon Jul 2 09:15:58 2012 : Info: ++++[eap] returns updated Mon Jul 2 09:15:58 2012 : Info: [files] users: Matched entry testtest at line 1 Mon Jul 2 09:15:58 2012 : Info: ++++[files] returns ok Mon Jul 2 09:15:58 2012 : Info: ++++[expiration] returns noop Mon Jul 2 09:15:58 2012 : Info: ++++[logintime] returns noop Mon Jul 2 09:15:58 2012 : Info: [pap] WARNING: Auth-Type already set. Not setting to PAP Mon Jul 2 09:15:58 2012 : Info: ++++[pap] returns noop Mon Jul 2 09:15:58 2012 : Info: +++- else else returns updated Mon Jul 2 09:15:58 2012 : Info: ++- else else returns updated Mon Jul 2 09:15:58 2012 : Info: Found Auth-Type = EAP Mon Jul 2 09:15:58 2012 : Info: # Executing group from file /etc/freeradius/sites-enabled/default Mon Jul 2 09:15:58 2012 : Info: +- entering group authenticate {...} Mon Jul 2 09:15:58 2012 : Info: [eap] EAP Identity Mon Jul 2 09:15:58 2012 : Info: [eap] processing type tls Mon Jul 2 09:15:58 2012 : Info: [tls] Initiate Mon Jul 2 09:15:58 2012 : Info: [tls] Start returned 1 Mon Jul 2 09:15:58 2012 : Info: ++[eap] returns handled Sending Access-Challenge of id 199 to 192.168.61.11 port 3072 EAP-Message = 0x010200061920 Message-Authenticator = 0x00000000000000000000000000000000 State = 0x85469e2a854487589fb1196910cb8ae3 Mon Jul 2 09:15:58 2012 : Info: Finished request 125. Mon Jul 2 09:15:58 2012 : Debug: Going to the next request Mon Jul 2 09:15:58 2012 : Debug: Waking up in 2.4 seconds. After that it repeats the login attempt and at some point tries to authenticate against ActiveDirectory with ntlm, which doesn't work since the user exists only in the users file. Can someone help me out here? Thanks. PS: Hope this helps, freeradius trying to auth against AD: Mon Jul 2 09:15:58 2012 : Info: ++[chap] returns noop Mon Jul 2 09:15:58 2012 : Info: ++[mschap] returns noop Mon Jul 2 09:15:58 2012 : Info: [suffix] No '@' in User-Name = "testtest", looking up realm NULL Mon Jul 2 09:15:58 2012 : Info: [suffix] Found realm "NULL" Mon Jul 2 09:15:58 2012 : Info: [suffix] Adding Stripped-User-Name = "testtest" Mon Jul 2 09:15:58 2012 : Info: [suffix] Adding Realm = "NULL" Mon Jul 2 09:15:58 2012 : Info: [suffix] Authentication realm is LOCAL. Mon Jul 2 09:15:58 2012 : Info: ++[suffix] returns ok Mon Jul 2 09:15:58 2012 : Info: ++[control] returns ok Mon Jul 2 09:15:58 2012 : Info: [eap] EAP packet type response id 7 length 67 Mon Jul 2 09:15:58 2012 : Info: [eap] No EAP Start, assuming it's an on-going EAP conversation Mon Jul 2 09:15:58 2012 : Info: ++[eap] returns updated Mon Jul 2 09:15:58 2012 : Info: [files] users: Matched entry testtest at line 1 Mon Jul 2 09:15:58 2012 : Info: ++[files] returns ok Mon Jul 2 09:15:58 2012 : Info: ++[smbpasswd] returns notfound Mon Jul 2 09:15:58 2012 : Info: ++[expiration] returns noop Mon Jul 2 09:15:58 2012 : Info: ++[logintime] returns noop Mon Jul 2 09:15:58 2012 : Info: [pap] WARNING: Auth-Type already set. Not setting to PAP Mon Jul 2 09:15:58 2012 : Info: ++[pap] returns noop Mon Jul 2 09:15:58 2012 : Info: Found Auth-Type = EAP Mon Jul 2 09:15:58 2012 : Info: # Executing group from file /etc/freeradius/sites-enabled/inner-tunnel Mon Jul 2 09:15:58 2012 : Info: +- entering group authenticate {...} Mon Jul 2 09:15:58 2012 : Info: [eap] Request found, released from the list Mon Jul 2 09:15:58 2012 : Info: [eap] EAP/mschapv2 Mon Jul 2 09:15:58 2012 : Info: [eap] processing type mschapv2 Mon Jul 2 09:15:58 2012 : Info: [mschapv2] # Executing group from file /etc/freeradius/sites-enabled/inner-tunnel Mon Jul 2 09:15:58 2012 : Info: [mschapv2] +- entering group MS-CHAP {...} Mon Jul 2 09:15:58 2012 : Info: [mschap] Creating challenge hash with username: testtest Mon Jul 2 09:15:58 2012 : Info: [mschap] Told to do MS-CHAPv2 for testtest with NT-Password Mon Jul 2 09:15:58 2012 : Info: [mschap] expand: --username=%{mschap:User-Name:-None} -> --username=testtest Mon Jul 2 09:15:58 2012 : Info: [mschap] No NT-Domain was found in the User-Name. Mon Jul 2 09:15:58 2012 : Info: [mschap] expand: %{mschap:NT-Domain} -> Mon Jul 2 09:15:58 2012 : Info: [mschap] ... expanding second conditional Mon Jul 2 09:15:58 2012 : Info: [mschap] expand: --domain=%{%{mschap:NT-Domain}:-AD.CXO.NAME} -> --domain=AD.CXO.NAME Mon Jul 2 09:15:58 2012 : Info: [mschap] mschap2: 82 Mon Jul 2 09:15:58 2012 : Info: [mschap] Creating challenge hash with username: testtest Mon Jul 2 09:15:58 2012 : Info: [mschap] expand: --challenge=%{mschap:Challenge:-00} -> --challenge=dd441972f987d68b Mon Jul 2 09:15:58 2012 : Info: [mschap] expand: --nt-response=%{mschap:NT-Response:-00} -> --nt-response=7e6c537cd5c26093789cf7831715d378e16ea3e6c5b1f579 Mon Jul 2 09:15:58 2012 : Debug: Exec-Program output: Logon failure (0xc000006d) Mon Jul 2 09:15:58 2012 : Debug: Exec-Program-Wait: plaintext: Logon failure (0xc000006d) Mon Jul 2 09:15:58 2012 : Debug: Exec-Program: returned: 1 Mon Jul 2 09:15:58 2012 : Info: [mschap] External script failed. Mon Jul 2 09:15:58 2012 : Info: [mschap] FAILED: MS-CHAP2-Response is incorrect Mon Jul 2 09:15:58 2012 : Info: ++[mschap] returns reject Mon Jul 2 09:15:58 2012 : Info: [eap] Freeing handler Mon Jul 2 09:15:58 2012 : Info: ++[eap] returns reject Mon Jul 2 09:15:58 2012 : Info: Failed to authenticate the user. Mon Jul 2 09:15:58 2012 : Auth: Login incorrect (mschap: External script says Logon failure (0xc000006d)): [testtest] (from client techap01 port 0 via TLS tunnel) PPS: Maybe the problem is located here: In /etc/freeradius/modules/ntlm_auth I have set ntlm to: program = "/usr/bin/ntlm_auth --request-nt-key --domain=AD.CXO.NAME --username=%{mschap:User-Name} --password=%{User-Password}" I need this, so users can login without adding @ad.cxo.name to their usernames. But how can I tell freeradius to try both logins, [email protected] (should fail) testtest (against users file - should work)

    Read the article

  • Why do we get a sudden spike in response times?

    - by Christian Hagelid
    We have an API that is implemented using ServiceStack which is hosted in IIS. While performing load testing of the API we discovered that the response times are good but that they deteriorate rapidly as soon as we hit about 3,500 concurrent users per server. We have two servers and when hitting them with 7,000 users the average response times sit below 500ms for all endpoints. The boxes are behind a load balancer so we get 3,500 concurrents per server. However as soon as we increase the number of total concurrent users we see a significant increase in response times. Increasing the concurrent users to 5,000 per server gives us an average response time per endpoint of around 7 seconds. The memory and CPU on the servers are quite low, both while the response times are good and when after they deteriorate. At peak with 10,000 concurrent users the CPU averages just below 50% and the RAM sits around 3-4 GB out of 16. This leaves us thinking that we are hitting some kind of limit somewhere. The below screenshot shows some key counters in perfmon during a load test with a total of 10,000 concurrent users. The highlighted counter is requests/second. To the right of the screenshot you can see the requests per second graph becoming really erratic. This is the main indicator for slow response times. As soon as we see this pattern we notice slow response times in the load test. How do we go about troubleshooting this performance issue? We are trying to identify if this is a coding issue or a configuration issue. Are there any settings in web.config or IIS that could explain this behaviour? The application pool is running .NET v4.0 and the IIS version is 7.5. The only change we have made from the default settings is to update the application pool Queue Length value from 1,000 to 5,000. We have also added the following config settings to the Aspnet.config file: <system.web> <applicationPool maxConcurrentRequestsPerCPU="5000" maxConcurrentThreadsPerCPU="0" requestQueueLimit="5000" /> </system.web> More details: The purpose of the API is to combine data from various external sources and return as JSON. It is currently using an InMemory cache implementation to cache individual external calls at the data layer. The first request to a resource will fetch all data required and any subsequent requests for the same resource will get results from the cache. We have a 'cache runner' that is implemented as a background process that updates the information in the cache at certain set intervals. We have added locking around the code that fetches data from the external resources. We have also implemented the services to fetch the data from the external sources in an asynchronous fashion so that the endpoint should only be as slow as the slowest external call (unless we have data in the cache of course). This is done using the System.Threading.Tasks.Task class. Could we be hitting a limitation in terms of number of threads available to the process?

    Read the article

  • django, mod_wsgi, MySQL High CPU - Problems

    - by Red Rover
    I am having a problem with an OSQA site. It is Django/Apache/mod_wsgi configured site. Every hour, the CPU spikes to 164% (Average) for task HTTPD. After 10 minutes, it frees back up. I have reviewed the logs, cron tables, made many config changes, but cannot track this problem down. Can someone please look at the information below and let me know if it is a configuration problem, or if anyone else has experienced this issue. Running TOP shows HTTPD using 165% of CPU VMware performance monitor also displays spikes. This happens every hour for 10 minutes. I have the following information from server status Server Version: Apache/2.2.15 (Unix) DAV/2 mod_wsgi/3.2 Python/2.6.6 Server Built: Feb 7 2012 09:50:15 Current Time: Sunday, 10-Jun-2012 21:44:29 EDT Restart Time: Sunday, 10-Jun-2012 19:44:51 EDT Parent Server Generation: 0 Server uptime: 1 hour 59 minutes 37 seconds Total accesses: 1088 - Total Traffic: 11.5 MB CPU Usage: u80.26 s243.8 cu0 cs0 - 4.52% CPU load .152 requests/sec - 1682 B/second - 10.8 kB/request 4 requests currently being processed, 11 idle workers ....._..........__......W....................................... ...................................C._..._....._L__._L_._....... ...................... Scoreboard Key: "_" Waiting for Connection, "S" Starting up, "R" Reading Request, "W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup, "C" Closing connection, "L" Logging, "G" Gracefully finishing, "I" Idle cleanup of worker, "." Open slot with no current process Srv PID Acc M CPU SS Req Conn Child Slot Client VHost Request 0-0 - 0/0/34 . 0.42 327 17 0.0 0.00 0.67 127.0.0.1 osqa.informs.org OPTIONS * HTTP/1.0 1-0 - 0/0/22 . 0.31 339 32 0.0 0.00 0.26 127.0.0.1 osqa.informs.org OPTIONS * HTTP/1.0 2-0 - 0/0/22 . 0.65 358 10 0.0 0.00 0.31 127.0.0.1 osqa.informs.org OPTIONS * HTTP/1.0 3-0 - 0/0/31 . 1.03 378 31 0.0 0.00 0.60 127.0.0.1 osqa.informs.org OPTIONS * HTTP/1.0 4-0 - 0/0/20 . 0.45 356 9 0.0 0.00 0.31 127.0.0.1 osqa.informs.org OPTIONS * HTTP/1.0 5-0 18852 0/16/34 _ 0.98 27 18120 0.0 0.37 0.62 69.180.250.36 osqa.informs.org GET /questions/289/what-is-the-difference-between-operations-re 6-0 - 0/0/32 . 0.94 309 29 0.0 0.00 0.64 127.0.0.1 osqa.informs.org OPTIONS * HTTP/1.0 7-0 - 0/0/31 . 1.15 382 32 0.0 0.00 0.75 127.0.0.1 osqa.informs.org OPTIONS * HTTP/1.0 8-0 - 0/0/21 . 0.28 403 19 0.0 0.00 0.20 127.0.0.1 osqa.informs.org OPTIONS * HTTP/1.0 9-0 - 0/0/32 . 1.37 288 16 0.0 0.00 0.60 127.0.0.1 osqa.informs.org OPTIONS * HTTP/1.0 10-0 - 0/0/33 . 1.72 383 16 0.0 0.00 0.40 127.0.0.1 osqa.informs.org OPTIONS * HTTP/1.0 I am running Django 1.3 This is a mod_wsgi configuration and copied is the wsgi.conf file: <IfModule !python_module> <IfModule !wsgi_module> LoadModule wsgi_module modules/mod_wsgi.so <IfModule wsgi_module> <Directory /var/www/osqa> Order allow,deny Allow from all #Deny from all </Directory> WSGISocketPrefix /var/run/wsgi WSGIPythonEggs /var/tmp WSGIDaemonProcess OSQA maximum-requests=10000 WSGIProcessGroup OSQA Alias /admin_media/ /usr/lib/python2.6/site-packages/Django-1.2.5-py2.6.egg/django/contrib/admin/media/ Alias /m/ /var/www/osqa/forum/skins/ Alias /upfiles/ /var/www/osqa/forum/upfiles/ <Directory /var/www/osqa/forum/skins> Order allow,deny Allow from all </Directory> WSGIScriptAlias / /var/www/osqa/osqa.wsgi </IfModule> </IfModule> </IfModule> This is the httpd.conf file Timeout 120 KeepAlive Off MaxKeepAliveRequests 100 MaxKeepAliveRequests 400 KeepAliveTimeout 3 <IfModule prefork.c> Startservers 15 MinSpareServers 10 MaxSpareServers 20 ServerLimit 50 MaxClients 50 MaxRequestsPerChild 0 </IfModule> <IfModule worker.c> StartServers 4 MaxClients 150 MinSpareThreads 25 MaxSpareThreads 75 ThreadsPerChild 25 MaxRequestsPerChild 0 </IfModule> We are using MySQL The server is an ESX4i, configured for the VM to use 4 CPUs and 8 GB Ram. Hyper threading is enabled, 2 physical CPU's, with 4 Logical. the CPU are Intel Xeon 2.8 GHz. Total memory is 12GB

    Read the article

  • Logging to MySQL without empty rows/skipped records?

    - by Lee Ward
    I'm trying to figure out how to make Squid proxy log to MySQL. I know ACL order is pretty important but I'm not sure if I understand exactly what ACLs are or do, it's difficult to explain, but hopefully you'll see where I'm going with this as you read! I have created the lines to make Squid interact with a helper in squid.conf as follows: external_acl_type mysql_log %LOGIN %SRC %PROTO %URI php /etc/squid3/custom/mysql_lg.php acl ex_log external mysql_log http_access allow ex_log The external ACL helper (mysql_lg.php) is a PHP script and is as follows: error_reporting(0); if (! defined(STDIN)) { define("STDIN", fopen("php://stdin", "r")); } $res = mysql_connect('localhost', 'squid', 'testsquidpw'); $dbres = mysql_select_db('squid', $res); while (!feof(STDIN)) { $line = trim(fgets(STDIN)); $fields = explode(' ', $line); $user = rawurldecode($fields[0]); $cli_ip = rawurldecode($fields[1]); $protocol = rawurldecode($fields[2]); $uri = rawurldecode($fields[3]); $q = "INSERT INTO logs (id, user, cli_ip, protocol, url) VALUES ('', '".$user."', '".$cli_ip."', '".$protocol."', '".$uri."');"; mysql_query($q) or die (mysql_error()); if ($fault) { fwrite(STDOUT, "ERR\n"); }; fwrite(STDOUT, "OK\n"); } The configuration I have right now looks like this: ## Authentication Handler auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp auth_param ntlm children 30 auth_param negotiate program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic auth_param negotiate children 5 # Allow squid to update log external_acl_type mysql_log %LOGIN %SRC %PROTO %URI php /etc/squid3/custom/mysql_lg.php acl ex_log external mysql_log http_access allow ex_log acl localnet src 172.16.45.0/24 acl AuthorizedUsers proxy_auth REQUIRED acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl CONNECT method CONNECT acl blockeddomain url_regex "/etc/squid3/bl.acl" http_access deny blockeddomain deny_info ERR_BAD_GENERAL blockeddomain # Deny requests to certain unsafe ports http_access deny !Safe_ports # Deny CONNECT to other than secure SSL ports http_access deny CONNECT !SSL_ports # Allow the internal network access to this proxy http_access allow localnet # Allow authorized users access to this proxy http_access allow AuthorizedUsers # FINAL RULE - Deny all other access to this proxy http_access deny all From testing, the closer to the bottom I place the logging lines the less it logs. Oftentimes, it even places empty rows in to the MySQL table. The file-based logs in /var/log/squid3/access.log are correct but many of the rows in the access logs are missing from the MySQL logs. I can't help but think it's down to the order I'm putting lines in because I want to log everything to MySQL, unauthenticated requests, blocked requests, which category blocked a specific request. The reason I want this in MySQL is because I'm trying to have everything managed via a custom web-based frontend and want to avoid using any shell commands and access to system log files if I can help it. The end result is to make it as easy as possible to maintain without keeping staff waiting on the phone whilst I add a new rule and reload the server! Hopefully someone can help me out here because this is very much a learning experience for me and I'm pretty stumped. Many thanks in advance for any help!

    Read the article

  • arp problems with transparent bridge on linux

    - by Mink
    I've been trying to secure my virtual machines on my esx server by putting them behind a transparent bridge with 2 interfaces, one in front, one at the back. My intention is to put all the firewall rules in one place (instead of on each virtual server). I've been using as bridge a blank new virtual machine based on arch linux (but I suspect it doesn't matter which brand of linux it is). What I have is 2 virtual switchs (thus two Virtual Network, VN_front and VN_back), each with 2 types of ports (switched/separated or promiscious/where the machine can see all packets). On my bridge machine, I've set up 2 virtual NIC, one on VN_front, one on VN_back, both in promisc mode. I've created a bridge br0 with both NIC in it: brctl addbr br0 brctl stp br0 off brctl addif br0 front_if brctl addif br0 back_if Then brought them up: ifconfig front_if 0.0.0.0 promisc ifconfig back_if 0.0.0.0 promisc ifconfig br0 0.0.0.0 (I use promisc mode, because I'm not sure I can do without, thinking that maybe the packets don't reach the NICs) Then I took one of my virtual server sitting on VN_front, and plugged it to VN_back instead (that's the nifty use case I'm thinking about, being able to move my servers around just by changing the VN they are plugged into, without changing anything in the configuration). Then I looked into the macs "seen" by my addressless bridge using brctl showmacs br0 and it did show my server from both sides: I get something that looks like this : port no mac addr is local? ageing timer 2 00:0c:29:e1:54:75 no 9.27 1 00:0c:29:fd:86:0c no 9.27 2 00:50:56:90:05:86 no 73.38 1 00:50:56:90:05:88 no 0.10 2 00:50:56:90:05:8b yes 0.00 << FRONT VN 1 00:50:56:90:05:8c yes 0.00 << BACK VN 2 00:50:56:90:19:18 no 13.55 2 00:50:56:90:3c:cf no 13.57 the thing is that the server that are plugged in front/back are not shown on the correct port. I suspect some horrible thing happening in the ARP-world... :-/ If I ping from a front virtual server to a back virtual server, I can only see the back machine if that back machine pings something in the front. As soon as I stop the ping from the back machine, the ping from the front machine stops getting through... I've noticed that if the back machine pings, then its port on the bridge is the correct one... I've tried to play with the arp_ switch of /proc/sys, but with no clear effect on the end result... /proc/sys/net/ipv4/ip_forward doesn't seem to be of any use when using a bridge (seems it's all taken care of by brctl) /proc/sys/net/ipv4/conf//arp_ don't seem to change much either... (tried arp_announce to 2 or 8 - like suggested elsewhere - and arp_ignore to 0 or 1 ) All the examples I've seen have a different subnet on either side like 10.0.1.0/24 and 10.0.2.0/24... In my case I want 10.0.1.0/24 on both side (just like a transparent switch - except it's a hidden fw ). Turning stp on/off doesn't seem to have any impact on my issue. It's as if the arp packets where getting through the bridge, corrupting the other side with false data... I've tried to use the -arp on each interface, br0, front, back... it breaks the thing altogether... I suspect it has something to do with both side being on the same subnet... I've thought about putting all my machine behind the fw, so as to have all the same subnet at the back... but I'm stuck with my provider's gateway standing at the front with part of my subnet (in fact 3 appliance to route the whole subnet), so I'll always have ips from the same subnet on both side, whatever I do... (I'm using fixed front IPs on my delegated subnet). I'm at a loss... -_-'' Thx for your help. (As anyone tried something like this? from within ESXi?) (It's not just a stunt, the idea is to have something like fail2ban running on some servers, sending their banned IP to the bridge/fw so that it too could ban them - saving all the other servers from that same attacker in one go, allowing for some honeypot that would trigger the fw from any kind of suitable response, and stuffs of the sort... I am aware I could use something like snort, but it addresses some completely different kind of problems, in a completely different way... )

    Read the article

  • Vagrant (Virtualbox) host-only multiple node networking issue

    - by Lorin Hochstein
    I'm trying to use a multi-VM vagrant environment as a testbed for deploying OpenStack, and I've run into a networking problem with trying to communicate from one VM, to a VM-inside-of-a-VM. I have two Vagrant nodes, a cloud controller node and a compute node. I'm using host-only networking. My Vagrantfile looks like this: Vagrant::Config.run do |config| config.vm.box = "precise64" config.vm.define :controller do |controller_config| controller_config.vm.network :hostonly, "192.168.206.130" # eth1 controller_config.vm.network :hostonly, "192.168.100.130" # eth2 controller_config.vm.host_name = "controller" end config.vm.define :compute1 do |compute1_config| compute1_config.vm.network :hostonly, "192.168.206.131" # eth1 compute1_config.vm.network :hostonly, "192.168.100.131" # eth2 compute1_config.vm.host_name = "compute1" compute1_config.vm.customize ["modifyvm", :id, "--memory", 1024] end end When I try to start up a (QEMU-based) VM, it boots successfully on compute1, and its virtual nic (vnet0) is connected via a bridge, br100: root@compute1:~# brctl show 100 bridge name bridge id STP enabled interfaces br100 8000.08002798c6ef no eth2 vnet0 When the QEMU VM makes a request to the DHCP server (dnsmasq) running on controller, I can see the request reaches the controller because of the output on the syslog on the controller: Aug 6 02:34:56 precise64 dnsmasq-dhcp[12042]: DHCPDISCOVER(br100) fa:16:3e:07:98:11 Aug 6 02:34:56 precise64 dnsmasq-dhcp[12042]: DHCPOFFER(br100) 192.168.100.2 fa:16:3e:07:98:11 However, the DHCPOFFER never makes it back to the VM running on compute1. If I watch the requests using tcpdump on the vboxnet3 interface on my host machine that runs Vagrant (Mac OS X), I can see both the requests and the replies $ sudo tcpdump -i vboxnet3 -n port 67 or port 68 tcpdump: WARNING: vboxnet3: That device doesn't support promiscuous mode (BIOCPROMISC: Operation not supported on socket) tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vboxnet3, link-type EN10MB (Ethernet), capture size 65535 bytes 22:51:20.694040 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:07:98:11, length 280 22:51:20.694057 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:07:98:11, length 280 22:51:20.696047 IP 192.168.100.1.67 > 192.168.100.2.68: BOOTP/DHCP, Reply, length 311 22:51:23.700845 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:07:98:11, length 280 22:51:23.700876 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:07:98:11, length 280 22:51:23.701591 IP 192.168.100.1.67 > 192.168.100.2.68: BOOTP/DHCP, Reply, length 311 22:51:26.705978 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:07:98:11, length 280 22:51:26.705995 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:07:98:11, length 280 22:51:26.706527 IP 192.168.100.1.67 > 192.168.100.2.68: BOOTP/DHCP, Reply, length 311 But, if I tcpdump on eth2 on compute, I only see the requests, not the replies: root@compute1:~# tcpdump -i eth2 -n port 67 or port 68 tcpdump: WARNING: eth2: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes 02:51:20.240672 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:07:98:11, length 280 02:51:23.249758 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:07:98:11, length 280 02:51:26.258281 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:07:98:11, length 280 At this point, I'm stuck. I'm not sure why the DHCP replies aren't making it to the compute node. Perhaps it has something to do with the configuration of the VirtualBox virtual switch/router? Note that eth2 interfaces on both nodes have been set to promiscuous mode.

    Read the article

  • Why the server is not responding?

    - by par
    Hello! Our server occasionally refuses to serve a simple HTML page. This is happening during a relatively high number of requests. However, the processor is not heavy loaded and there are a lot of free memory. The error seems to occure 1 out of 50 requests in average, depending on the server load. I need to find the source of the problem and take the appropriate actions to eliminate it. I have a suspicion that the problem source is a huge number of incoming network packets. There are 5000 packets per second on average. Traffic - 2 MBits/sec Can this be the cause of the error? There is an interesting thing, in case the server fails to respond, the request string is not logged to access.log by Apache. The error is repeatable from several client computers. DNS is not involved, since I have accessed the server by the IP. I have profiled the problem case with tcpdump utility. These are the good and bad sessions traced by tcpdump. The request is the same in both experiments. Good - server returns response. Bad - no response, time-out error. ---- Bad ---- 12:23:36.366292 IP 123.45.67.890.61749 > myserver.superbservers.com.www: S 2125316338:2125316338(0) win 8192 <mss 1460,nop,wscale 2,nop,nop,sackOK> 12:23:39.362394 IP 123.45.67.890.61749 > myserver.superbservers.com.www: S 2125316338:2125316338(0) win 8192 <mss 1460,nop,wscale 2,nop,nop,sackOK> 12:23:45.365567 IP 123.45.67.890.61749 > myserver.superbservers.com.www: S 2125316338:2125316338(0) win 8192 <mss 1460,nop,nop,sackOK> -------- ---- Good ---- 12:27:07.632229 IP 123.45.67.890.63914 > myserver.superbservers.com.www: S 3581365570:3581365570(0) win 8192 <mss 1460,nop,wscale 2,nop,nop,sackOK> 12:27:10.620946 IP 123.45.67.890.63914 > myserver.superbservers.com.www: S 3581365570:3581365570(0) win 8192 <mss 1460,nop,wscale 2,nop,nop,sackOK> 12:27:10.620969 IP myserver.superbservers.com.www > 123.45.67.890.63914: S 2654770980:2654770980(0) ack 3581365571 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 6> 12:27:10.838747 IP 123.45.67.890.63914 > myserver.superbservers.com.www: . ack 1 win 4380 12:27:10.957143 IP 123.45.67.890.63914 > myserver.superbservers.com.www: P 1:213(212) ack 1 win 4380 12:27:10.957152 IP myserver.superbservers.com.www > 123.45.67.890.63914: . ack 213 win 108 12:27:10.965543 IP myserver.superbservers.com.www > 123.45.67.890.63914: P 1:630(629) ack 213 win 108 12:27:10.965621 IP myserver.superbservers.com.www > 123.45.67.890.63914: F 630:630(0) ack 213 win 108 12:27:11.183540 IP 123.45.67.890.63914 > myserver.superbservers.com.www: . ack 631 win 4222 12:27:11.185657 IP 123.45.67.890.63914 > myserver.superbservers.com.www: F 213:213(0) ack 631 win 4222 12:27:11.185663 IP myserver.superbservers.com.www > 123.45.67.890.63914: . ack 214 win 108 -------- Hoster: SuperbHosting OS: Ubuntu Server parameters: E6300 CONROE 1.86GHZ 2 X 1MB CACHE 1066 1GB DDR2 667MHZ This is a link to apache configuration file we use http://repkin5.snow.prohosting.com/apache.txt This is server-status report taken right after time-out error. http://repkin5.snow.prohosting.com/server-status.htm There are only 10 Child Servers running out of 120, so enough space for new requests. VMSTAT procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 0 0 8900 725900 8468 65684 0 0 5 18 11 33 4 3 92 1

    Read the article

  • Apache reverse proxy POST 403

    - by qkslvrwolf
    I am trying to get Jira and Stash to talk to each other via a Trusted Application link. The setup, currently, looks like this: Jira - http - Jira Proxy -https- stash proxy -http- stash. Jira and the Jira proxy are on the same machine. The Jira Proxy is showing 403 Forbidden for POST requests from the stash server. It works (or seems to ) for everything else. I contend that since we're seeing 403 forbiddens in the access log for apache, Jira is never seeing the request. Why is apache forbidding posts,and how do I fix it? Note that the IPs for both Stash and the Stash Proxy are in the "trusted host" section. My config: LogLevel info CustomLog "|/usr/sbin/rotatelogs /var/log/apache2/access.log 86400" common ServerSignature off ServerTokens prod Listen 8443 <VirtualHost *:443> ServerName jira.company.com SSLEngine on SSLOptions +StrictRequire SSLCertificateFile /etc/ssl/certs/server.cer SSLCertificateKeyFile /etc/ssl/private/server.key SSLProtocol +SSLv3 +TLSv1 SSLCipherSuite DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:AES256-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:AES128-SHA:EDH-RSA-DES-CBC3-SHA:EDH-DSS-DES-CBC3-SHA:DES-CBC3-SHA # If context path is not "/wiki", then send to /jira. RedirectMatch 301 ^/$ https://jira.company.com/jira RedirectMatch 301 ^/gsd(.*)$ https://jira.company.com/jira$1 ProxyRequests On ProxyPreserveHost On ProxyVia On ProxyPass /jira http://localhost:8080/jira ProxyPassReverse /jira http://localhost:8080/jira <Proxy *> Order deny,allow Allow from all </Proxy> RewriteEngine on RewriteLog "/var/log/apache2/rewrite.log" RewriteLogLevel 2 # Disable TRACE/TRACK requests, per security. RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK) RewriteRule .* - [F] DocumentRoot /var/www DirectoryIndex index.html <Directory /var/www> Options FollowSymLinks AllowOverride None Order deny,allow Allow from all </Directory> <LocationMatch "/"> Order deny,allow Deny from all allow from x.x.71.8 allow from x.x.8.123 allow from x.x.120.179 allow from x.x.120.73 allow from x.x.120.45 satisfy any SetEnvif Remote_Addr "x.x.71.8" TRUSTED_HOST SetEnvif Remote_Addr "x.x.8.123" TRUSTED_HOST SetEnvif Remote_Addr "x.x.120.179" TRUSTED_HOST SetEnvif Remote_Addr "x.x.120.73" TRUSTED_HOST SetEnvif Remote_Addr "x.x.120.45" TRUSTED_HOST </LocationMatch> <LocationMatch ^> SSLRequireSSL AuthType CompanyNet PubcookieInactiveExpire -1 PubcookieAppID jira.company.com require valid-user RequestHeader set userid %{REMOTE_USER}s </LocationMatch> </VirtualHost> # Port open for SSL, non-pubcookie access. Used to access APIs with Basic Auth. <VirtualHost *:8443> SSLEngine on SSLOptions +StrictRequire SSLCertificateFile /etc/ssl/certs/server.cer SSLCertificateKeyFile /etc/ssl/private/server.key SSLProtocol +SSLv3 +TLSv1 SSLCipherSuite DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:AES256-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:AES128-SHA:EDH-RSA-DES-CBC3-SHA:EDH-DSS-DES-CBC3-SHA:DES-CBC3-SHA ProxyRequests On ProxyPreserveHost On ProxyVia On ProxyPass /jira http://localhost:8080/jira ProxyPassReverse /jira http://localhost:8080/jira <Proxy *> Order deny,allow Allow from all </Proxy> RewriteEngine on RewriteLog "/var/log/apache2/rewrite.log" RewriteLogLevel 2 # Disable TRACE/TRACK requests, per security. RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK) RewriteRule .* - [F] DocumentRoot /var/www DirectoryIndex index.html <Directory /var/www> Options FollowSymLinks AllowOverride None Order deny,allow Allow from all </Directory> </VirtualHost> <VirtualHost jira.company.com:80> ServerName jira.company.com RedirectMatch 301 /(.*)$ https://jira.company.com/$1 RewriteEngine on RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK) RewriteRule .* - [F] </VirtualHost> <VirtualHost *:80> ServerName go.company.com RedirectMatch 301 /(.*)$ https://jira.company.com/$1 RewriteEngine on RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK) RewriteRule .* - [F] </VirtualHost>

    Read the article

  • Could not find rake-10.1.0 in any of the sources

    - by spuder
    I've got a ruby on rails application (gitlab) which is installed via puppet. Everything on the test system runs fine, but production generates an error about rake Running /home/git/gitlab-shell/bin/check Could not find rake-10.1.0 in any of the sources Run bundle install to install missing gems. Here is the full rake check: root@gitlab:/home/git# sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production Checking Environment ... Git configured for git user? ... yes Has python2? ... yes python2 is supported version? ... yes Checking Environment ... Finished Checking GitLab Shell ... GitLab Shell version >= 1.7.1 ? ... OK (1.7.1) Repo base directory exists? ... yes Repo base directory is a symlink? ... no Repo base owned by git:git? ... yes Repo base access is drwxrws---? ... yes update hook up-to-date? ... yes update hooks in repos are links: ... Could not find rake-10.1.0 in any of the sources Run `bundle install` to install missing gems. gitlab-shell self-check failed Try fixing it: Make sure GitLab is running; Check the gitlab-shell configuration file: sudo -u git -H editor /home/git/gitlab-shell/config.yml Please fix the error above and rerun the checks. Checking GitLab Shell ... Finished Checking Sidekiq ... Running? ... yes Number of Sidekiq processes ... 1 Checking Sidekiq ... Finished Checking GitLab ... Database config exists? ... yes Database is SQLite ... no All migrations up? ... yes GitLab config exists? ... yes GitLab config outdated? ... no Log directory writable? ... yes Tmp directory writable? ... yes Init script exists? ... yes Init script up-to-date? ... yes projects have namespace: ... Spencer Owen / bar ... yes Projects have satellites? ... Spencer Owen / bar ... can't create, repository is empty Redis version >= 2.0.0? ... yes Your git bin path is "/usr/bin/git" Git version >= 1.7.10 ? ... yes (1.8.4) Checking GitLab ... Finished The step 'gitlab-shell check' effectively runs the following command. If I run that command manually, everything passes. root@gitlab:/home/git/gitlab# sudo -u git -H /home/git/gitlab-shell/bin/check Check GitLab API access: OK Check directories and files: /home/git/repositories: OK /home/git/.ssh/authorized_keys: OK I have verified that rake is in fact installed root@gitlab:/home/git/gitlab# gem install rake -v 10.1.0 root@gitlab:/home/git/gitlab# bundle install root@gitlab:/home/git/gitlab# sudo -u git -H gem install rake -v 10.1.0 root@gitlab:/home/git/gitlab# sudo -u git -H bundle install Ruby is installed with update alternatives root@gitlab:/home/git/gitlab# sudo -u git -H ruby --version ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-linux] root@gitlab:/home/git/gitlab# sudo -u git -H ls -l `which ruby` lrwxrwxrwx 1 root root 22 Oct 8 20:26 /usr/bin/ruby -> /etc/alternatives/ruby root@gitlab:/home/git/gitlab# sudo -u git -H gem --version 2.1.10 root@gitlab:/home/git/gitlab# sudo -u git -H ls -l `which gem` lrwxrwxrwx 1 root root 21 Oct 10 20:50 /usr/bin/gem -> /etc/alternatives/gem I've tried the solution mentioned below, to allow shared gems http://stackoverflow.com/questions/19284914/bundle-exec-fails-with-could-not-find-rake-10-1-0-in-any-of-the-sources http://stackoverflow.com/questions/18978002/could-not-find-rake-with-bundle-exec root@gitlab:/home/git/gitlab# cat /home/git/gitlab/.bundle/config --- BUNDLE_FROZEN: '1' BUNDLE_PATH: vendor/bundle BUNDLE_WITHOUT: development:test:postgres BUNDLE_DISABLE_SHARED_GEMS: '1' I've exhausted google, so I'm hoping for someone more familiar with ruby to offer any ideas how to resolve the error. Could not find rake-10.1.0 in any of the sources

    Read the article

  • Linux Kernel crash mutex_lock_slowpath "blocked for more than 120 seconds". What to do?

    - by Roddick
    I have out-of-the box Debian Lenny with non-custom kernel 2.6.26-2-amd64. Brand new server that is used to 5% of it's potential, CPU and Disk-wise. Meaning it probably not crashing because of overload. every few days it freezes with hundreds of these messages in console log: : [284847.828428] INFO: task apache2:12473 blocked for more than 120 seconds. : [284847.868468] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. : [284847.912759] apache2 D ffff8101bc6b7ab0 0 12473 14358 : [284847.912763] ffff810160d5bc50 0000000000000082 ffff8101c0002e40 0000000000000000 : [284847.912766] ffff8101a7c42950 ffff810327d92810 ffff8101a7c42bd8 0000000400000044 : [284847.912770] ffff8101c0002e40 00000000000612d0 0000000000000000 00000040000612d0 : [284847.912773] Call Trace: : [284847.912786] [<ffffffff80429b0d>] __mutex_lock_slowpath+0x64/0x9b : [284847.912790] [<ffffffff80429972>] mutex_lock+0xa/0xb : [284847.912794] [<ffffffff802a20b9>] do_lookup+0x82/0x1c1 : [284847.912800] [<ffffffff802a4271>] __link_path_walk+0x87a/0xd19 : [284847.912805] [<ffffffff80295844>] kmem_getpages+0x96/0x15f : [284847.912808] [<ffffffff80295fb7>] ____cache_alloc_node+0x6d/0x106 : [284847.912814] [<ffffffff802a4756>] path_walk+0x46/0x8b : [284847.912819] [<ffffffff802a4a82>] do_path_lookup+0x158/0x1cf : [284847.912822] [<ffffffff802a3879>] getname+0x140/0x1a7 : [284847.912827] [<ffffffff802a53f1>] __user_walk_fd+0x37/0x4c : [284847.912831] [<ffffffff8029e381>] vfs_lstat_fd+0x18/0x47 : [284847.912840] [<ffffffff8029e3c9>] sys_newlstat+0x19/0x31 : [284847.912848] [<ffffffff8020beda>] system_call_after_swapgs+0x8a/0x8f Almost all traces has __mutex_lock_slowpath as top-level. Only some has different trace: : [284847.737386] INFO: task apache2:12472 blocked for more than 120 seconds. : [284847.777551] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. : [284847.824881] apache2 D ffff8101bc6b7ab0 0 12472 14358 : [284847.824886] ffff8101b9cc1c50 0000000000000086 ffffffffa0131e0a 0000000000000002 : [284847.824889] ffff8102e7454300 ffff810324c6cad0 ffff8102e7454588 0000000000000000 : [284847.824893] 0000000000000001 0000000000000296 0000000000000003 ffff8101b9cc1c58 : [284847.824896] Call Trace: : [284847.828403] [<ffffffffa0131e0a>] :ext3:__ext3_journal_dirty_metadata+0x1e/0x46 : [284847.828412] [<ffffffff80429b0d>] __mutex_lock_slowpath+0x64/0x9b : [284847.828418] [<ffffffff80429972>] mutex_lock+0xa/0xb : [284847.828421] [<ffffffff802a20b9>] do_lookup+0x82/0x1c1 : [284847.828427] [<ffffffff802a4271>] __link_path_walk+0x87a/0xd19 : [284847.828428] [<ffffffff80271296>] find_lock_page+0x1f/0x8a : [284847.828428] [<ffffffff80273182>] filemap_fault+0x1c2/0x33c : [284847.828428] [<ffffffff802a4756>] path_walk+0x46/0x8b : [284847.828428] [<ffffffff802a4a82>] do_path_lookup+0x158/0x1cf : [284847.828428] [<ffffffff802a3879>] getname+0x140/0x1a7 : [284847.828428] [<ffffffff802a53f1>] __user_walk_fd+0x37/0x4c : [284847.828428] [<ffffffff8029e381>] vfs_lstat_fd+0x18/0x47 : [284847.828428] [<ffffffff8029e3c9>] sys_newlstat+0x19/0x31 : [284847.828428] [<ffffffff8020beda>] system_call_after_swapgs+0x8a/0x8f kernel: [1912668.466347] INFO: task apache2:17984 blocked for more than 120 seconds. [1912668.507035] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. : [1912668.555165] apache2 D ffff8101c5637ba0 0 17984 17282 : [1912668.596752] ffff810166a7dd30 0000000000000086 0000000000000000 ffff810166a7dcd8 : [1912668.643341] ffff8101c563c880 ffff81024505f000 0000000000000002 ffff810166a7dd68 : [1912668.699566] 0000000000000086 00000000000cb1a0 0000000000000000 ffff81017f344d60 : [1912668.744773] Call Trace: : [1912668.761754] [<ffffffff8022a3ed>] pick_next_task_fair+0x6e/0x7a : [1912668.829311] [<ffffffff802be0e2>] bio_alloc_bioset+0x89/0xd9 : [1912668.861930] [<ffffffff8024ac3a>] getnstimeofday+0x39/0x98 : [1912668.897005] [<ffffffff802710f6>] sync_page+0x0/0x41 : [1912668.927868] [<ffffffff80429487>] io_schedule+0x5c/0x9e : [1912668.960286] [<ffffffff80271132>] sync_page+0x3c/0x41 : [1912668.991756] [<ffffffff804295fa>] __wait_on_bit_lock+0x36/0x66 : [1912669.031757] [<ffffffff802710e3>] __lock_page+0x5e/0x64 : [1912669.064191] [<ffffffff802461d3>] wake_bit_function+0x0/0x23 : [1912669.100100] [<ffffffff80281bc5>] handle_mm_fault+0x5e4/0x8de : [1912669.134531] [<ffffffff802461a5>] autoremove_wake_function+0x0/0x2e : [1912669.174623] [<ffffffff802aa108>] fcntl_setlk+0x1cf/0x291 : [1912669.210623] [<ffffffff802461a5>] autoremove_wake_function+0x0/0x2e : [1912669.246923] [<ffffffff802a677f>] sys_fcntl+0x280/0x2f7 After googling for "mutex_lock_slowpath" I can only find the Kernel mailing list discussions that this issue was introduced in some commit. Wthout reference to verison. Discussions as recent as Jan 25, 2011. The Kernel I am using is form Debian Lenny, year ago. What should I do? Is this bug even fixed in kernel? if it's such obvious bug why it happens so rarely? Should I download latest kernel from kernel.org and upgrade? Should I use Debian backports to install new "Approved" kernel? Am I missing something? What to do?

    Read the article

  • Squid - Logging to MySQL without empty rows/skipped records?

    - by Lee Ward
    I'm trying to figure out how to make Squid proxy log to MySQL. I know ACL order is pretty important but I'm not sure if I understand exactly what ACLs are or do, it's difficult to explain, but hopefully you'll see where I'm going with this as you read! I have created the lines to make Squid interact with a helper in squid.conf as follows: external_acl_type mysql_log %LOGIN %SRC %PROTO %URI php /etc/squid3/custom/mysql_lg.php acl ex_log external mysql_log http_access allow ex_log The external ACL helper (mysql_lg.php) is a PHP script and is as follows: error_reporting(0); if (! defined(STDIN)) { define("STDIN", fopen("php://stdin", "r")); } $res = mysql_connect('localhost', 'squid', 'testsquidpw'); $dbres = mysql_select_db('squid', $res); while (!feof(STDIN)) { $line = trim(fgets(STDIN)); $fields = explode(' ', $line); $user = rawurldecode($fields[0]); $cli_ip = rawurldecode($fields[1]); $protocol = rawurldecode($fields[2]); $uri = rawurldecode($fields[3]); $q = "INSERT INTO logs (id, user, cli_ip, protocol, url) VALUES ('', '".$user."', '".$cli_ip."', '".$protocol."', '".$uri."');"; mysql_query($q) or die (mysql_error()); if ($fault) { fwrite(STDOUT, "ERR\n"); }; fwrite(STDOUT, "OK\n"); } The configuration I have right now looks like this: ## Authentication Handler auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp auth_param ntlm children 30 auth_param negotiate program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic auth_param negotiate children 5 # Allow squid to update log external_acl_type mysql_log %LOGIN %SRC %PROTO %URI php /etc/squid3/custom/mysql_lg.php acl ex_log external mysql_log http_access allow ex_log acl localnet src 172.16.45.0/24 acl AuthorizedUsers proxy_auth REQUIRED acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl CONNECT method CONNECT acl blockeddomain url_regex "/etc/squid3/bl.acl" http_access deny blockeddomain deny_info ERR_BAD_GENERAL blockeddomain # Deny requests to certain unsafe ports http_access deny !Safe_ports # Deny CONNECT to other than secure SSL ports http_access deny CONNECT !SSL_ports # Allow the internal network access to this proxy http_access allow localnet # Allow authorized users access to this proxy http_access allow AuthorizedUsers # FINAL RULE - Deny all other access to this proxy http_access deny all From testing, the closer to the bottom I place the logging lines the less it logs. Oftentimes, it even places empty rows in to the MySQL table. The file-based logs in /var/log/squid3/access.log are correct but many of the rows in the access logs are missing from the MySQL logs. I can't help but think it's down to the order I'm putting lines in because I want to log everything to MySQL, unauthenticated requests, blocked requests, which category blocked a specific request. The reason I want this in MySQL is because I'm trying to have everything managed via a custom web-based frontend and want to avoid using any shell commands and access to system log files if I can help it. The end result is to make it as easy as possible to maintain without keeping staff waiting on the phone whilst I add a new rule and reload the server! Hopefully someone can help me out here because this is very much a learning experience for me and I'm pretty stumped. Many thanks in advance for any help!

    Read the article

< Previous Page | 132 133 134 135 136 137 138 139 140 141 142 143  | Next Page >