Search Results

Search found 181 results on 8 pages for 'redis'.

Page 4/8 | < Previous Page | 1 2 3 4 5 6 7 8  | Next Page >

  • Hiredis waiting for message

    - by Vivek Goel
    I am using hiredis C library to connect to redis server. I am not able to figure out how to wait for new messages after subscribing to new message. My code look like: signal(SIGPIPE, SIG_IGN ); struct event_base *base = event_base_new(); redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379); if (c->err) { /* Let *c leak for now... */ printf("Error: %s\n", c->errstr); return 1; } redisLibeventAttach(c, base); redisAsyncSetConnectCallback(c, connectCallback); redisAsyncSetDisconnectCallback(c, disconnectCallback); redisAsyncCommand(c, NULL, NULL, "SET key %b", argv[argc - 1], strlen(argv[argc - 1])); redisAsyncCommand(c, getCallback, (char*) "end-1", "GET key"); redisAsyncCommand(c, getCallback, (char*) "end-1", "SUBSCRIBE foo"); Now how to tell hiredis to wait for message on channel ?

    Read the article

  • Daily, Weekly and Monthly Page View Counter

    - by Jens Fahnenbruck
    I'm building a website with user generated content. On the home page I want to show a list of all created items, and I want to be able to sort them by a view counter. That's sound easy, but I want multiple counters. I want to know which was the most visited item in the last day, last week or last months or overall. My first Idea was to create 4 counter columns in the item's DB-Table. One for each of daily, weekly, monthly and overall, and the create a cron job, that clears the daily counter every 24 hours, the weekly counter every 7 days and so on. But my problem with this is, what happens if I want to know which was the most viewed item of the week, just after the weekly counter got cleared? What I need is an efficient way to create a continous counter, which got reduced for every page view that is too old, and increased for every new page view. Right now I'm thinking of a solution with the redis server, but I have no solution yet. I'm just looking for a general idea here, but FYI I'm developing this application in Ruby on Rails.

    Read the article

  • node.js / socket.io, cookies only working locally

    - by Ben Griffiths
    I'm trying to use cookie based sessions, however it'll only work on the local machine, not over the network. If I remove the session related stuff, it will however work just great over the network... You'll have to forgive the lack of quality code here, I'm just starting out with node/socket etc etc, and finding any clear guides is tough going, so I'm in n00b territory right now. Basically this is so far hacked together from various snippets with about 10% understanding of what I'm actually doing... The error I see in Chrome is: socket.io.js:1632GET http://192.168.0.6:8080/socket.io/1/?t=1334431940273 500 (Internal Server Error) Socket.handshake ------- socket.io.js:1632 Socket.connect ------- socket.io.js:1671 Socket ------- socket.io.js:1530 io.connect ------- socket.io.js:91 (anonymous function) ------- /socket-test/:9 jQuery.extend.ready ------- jquery.js:438 And in the console for the server I see: debug - served static content /socket.io.js debug - authorized warn - handshake error No cookie My server is: var express = require('express') , app = express.createServer() , io = require('socket.io').listen(app) , connect = require('express/node_modules/connect') , parseCookie = connect.utils.parseCookie , RedisStore = require('connect-redis')(express) , sessionStore = new RedisStore(); app.listen(8080, '192.168.0.6'); app.configure(function() { app.use(express.cookieParser()); app.use(express.session( { secret: 'YOURSOOPERSEKRITKEY', store: sessionStore })); }); io.configure(function() { io.set('authorization', function(data, callback) { if(data.headers.cookie) { var cookie = parseCookie(data.headers.cookie); sessionStore.get(cookie['connect.sid'], function(err, session) { if(err || !session) { callback('Error', false); } else { data.session = session; callback(null, true); } }); } else { callback('No cookie', false); } }); }); var users_count = 0; io.sockets.on('connection', function (socket) { console.log('New Connection'); var session = socket.handshake.session; ++users_count; io.sockets.emit('users_count', users_count); socket.on('something', function(data) { io.sockets.emit('doing_something', data['data']); }); socket.on('disconnect', function() { --users_count; io.sockets.emit('users_count', users_count); }); }); My page JS is: jQuery(function($){ var socket = io.connect('http://192.168.0.6', { port: 8080 } ); socket.on('users_count', function(data) { $('#client_count').text(data); }); socket.on('doing_something', function(data) { if(data == '') { window.setTimeout(function() { $('#target').text(data); }, 3000); } else { $('#target').text(data); } }); $('#textbox').keydown(function() { socket.emit('something', { data: 'typing' }); }); $('#textbox').keyup(function() { socket.emit('something', { data: '' }); }); });

    Read the article

  • Rails development environment Resque.enqueue does not create jobs

    - by anton evangelatov
    I am having the same problem like Rails custom environment Resque.enqueue does not create jobs , but the solution there doesn't work for me. I'm using Resque for a couple of asynchronous jobs. It works just fine for the staging environment, but for some reason it stopped working on development environment. For example, if I run the following: $ rails c development > Resque.enqueue(MyLovelyJob, 1) Nothing is enqueued. I check Resque using resque-web If I run it on staging - it works just fine. $ rails c staging > Resque.enqueue(MyLovelyJob, 1) I have tried to duplicate the 2 environment, and they seem to use absolutely the same configurations (database.yml , config/environment , etc.), but development is still not working. If I do > Resque.enqueue(UpdateInstancesData, 2) > => true > Resque.info > => { > :pending => 0, > :processed => 0, > :queues => 0, > :workers => 1, > :working => 0, > :failed => 0, > :servers => [ > [0] "redis://127.0.0.1:6379/0" > ], > :environment => "development" > } Any suggestions where to look in order to debug this? I am running the application via foreman. My Procfile looks like: faye: rackup faye.ru -s thin -E production worker1: bundle exec rake resque:work QUEUE=* VERBOSE=1 worker2: bundle exec rake resque:work QUEUE=* VERBOSE=1 clock: bundle exec rake resque:scheduler VERBOSE=1 web: bundle exec rails s For staging, as mentioned, everything works and the log from foreman is: 17:03:42 clock.1 | 2013-06-26 17:03:42 Reloading Schedule 17:03:42 clock.1 | 2013-06-26 17:03:42 Loading Schedule 17:03:42 clock.1 | 2013-06-26 17:03:42 Scheduling logging_test 17:03:42 clock.1 | 2013-06-26 17:03:42 Schedules Loaded 17:03:43 worker2.1 | *** Starting worker ttttt-mbp.local:69573:* 17:03:43 worker2.1 | *** Registered signals 17:03:43 worker2.1 | *** Running before_first_fork hooks 17:03:43 worker1.1 | *** Starting worker ttttt-mbp.local:69572:* 17:03:43 worker1.1 | *** Registered signals 17:03:43 worker2.1 | *** Checking another_queue 17:03:43 worker2.1 | *** Checking anotherqueue 17:03:43 worker2.1 | *** Checking statused 17:03:43 worker2.1 | *** Found job on statused 17:03:43 worker2.1 | *** got: (Job{statused} | LoggingTest | ["57e89a1c1b24ce6866bcf5d0e1c07f01", {}]) 17:06:30 clock.1 | 2013-06-26 17:06:30 queueing LoggingTest (logging_test) 17:06:33 worker1.1 | *** Checking another_queue 17:06:33 worker2.1 | *** Checking another_queue 17:06:33 worker1.1 | *** Checking anotherqueue 17:06:33 worker2.1 | *** Checking anotherqueue 17:06:33 worker1.1 | *** Found job on anotherqueue 17:06:33 worker1.1 | *** got: (Job{anotherqueue} | LoggingTest | ["0d976869a945766e0cfeca83e7349305", {}]) 17:06:33 worker1.1 | *** resque-1.24.1: Processing anotherqueue since 1372259193 [LoggingTest] 17:06:33 worker1.1 | *** Running before_fork hooks with [(Job{anotherqueue} | LoggingTest | ["0d976869a945766e0cfeca83e7349305", {}])] 17:06:33 worker1.1 | *** resque-1.24.1: Forked 69955 at 1372259193 17:06:33 worker2.1 | *** resque-1.24.1: Forked 69956 at 1372259193 17:06:33 worker1.1 | *** Running after_fork hooks with [(Job{anotherqueue} | LoggingTest | ["0d976869a945766e0cfeca83e7349305", {}])] 17:06:33 worker1.1 | JOB :: LoggingTest 17:06:33 worker1.1 | 55555 17:06:33 worker1.1 | *** done: (Job{anotherqueue} | LoggingTest | ["0d976869a945766e0cfeca83e7349305", {}]) whereas for development it doesn't seem to enqueue and then find the job. If there is a job already in the queue (pending, left over from staging environment) the workers from development don't process it. 17:01:23 clock.1 | 2013-06-26 17:01:23 Reloading Schedule 17:01:23 clock.1 | 2013-06-26 17:01:23 Loading Schedule 17:01:23 clock.1 | 2013-06-26 17:01:23 Scheduling logging_test 17:01:23 clock.1 | 2013-06-26 17:01:23 Scheduling update_instances_data 17:01:23 clock.1 | 2013-06-26 17:01:23 Schedules Loaded 17:03:10 clock.1 | 2013-06-26 17:03:10 queueing LoggingTest (logging_test) 17:03:14 worker1.1 | *** Checking another_queue 17:03:14 worker2.1 | *** Checking another_queue 17:03:14 worker1.1 | *** Checking anotherqueue 17:03:14 worker2.1 | *** Checking anotherqueue 17:03:14 worker1.1 | *** Checking statused 17:03:14 worker2.1 | *** Checking statused

    Read the article

  • Is there or why not having a ruby technology specification similar to Java's JSR?

    - by romeu.hcf
    I think on a community portal where specifications are made, documented and specified to reference libraries and systems implementation. An example: A specification for Message Queue where redis clients, for instance, could implement it and where the libraries could be validated by the specification's test suite. Redic, redis-rb, hiredis, redis-connection-pool, redis-namespace should all implement this specification. This way, being easily replaced.

    Read the article

  • NoSQL replacement for memcache

    - by Juan Antonio Gomez Moriano
    We are having a situation in which the values we store on memcache are bigger than 1MB. It is not possible to make such values smaller, and even if there was a way, we need to persist them to disk. One solution would be to recompile the memcache server to allow say 2MB values, but this is either not clean nor a complete solution (again, we need to persist the values). Good news is that We can predict quite acurately how many key/values pair we are going to have We can also predict the total size we will need. A key feature for us is the speed of memcache. So question is: is there any noSQL replacement for memcache which will allow us to have values longer than 1MB AND store them in disk without loss of speed? In the past I have used tokyotyrant/cabinet but seems to be deprecated now. Any idea?

    Read the article

  • rails foreman does not load all my services on start

    - by Rubytastic
    Rails foreman does not load all my services defined in Procfile. Procfile.rb: redis: redis-server resque: bundle exec rake resque:start &&> log/resque_worker_queue.log privpub: bundle exec rackup private_pub.ru -s thin -E production & &> log/private_pub.log sunspot: bundle exec rake sunspot:solr:run I always have to manually start all of them by copy paste the commands in terminal foreman start does not work, what am i missing? This is foreman output: 12:35:40 privpub.1 | process terminated 12:35:40 system | sending SIGTERM to all processes 12:35:40 system | sending SIGTERM to pid 4375 12:35:40 redis.1 | [4375] 02 Jun 12:35:40 # Received SIGTERM, scheduling shutdown... 12:35:40 redis.1 | [4375] 02 Jun 12:35:40 # User requested shutdown... 12:35:40 redis.1 | [4375] 02 Jun 12:35:40 * Saving the final RDB snapshot before exiting. 12:35:40 redis.1 | [4375] 02 Jun 12:35:40 * DB saved on disk 12:35:40 redis.1 | [4375] 02 Jun 12:35:40 # Redis is now ready to exit, bye bye... 12:35:40 system | sending SIGTERM to pid 4376 12:35:40 resque.1 | rake aborted! 12:35:40 resque.1 | SIGTERM 12:35:40 resque.1 | 12:35:40 resque.1 | (See full trace by running task with --trace) 12:35:40 system | sending SIGTERM to pid 4378 12:35:40 sunspot.1 | rake aborted! 12:35:40 sunspot.1 | SIGTERM 12:35:40 sunspot.1 | 12:35:40 sunspot.1 | (See full trace by running task with --trace) 12:35:40 sunspot.1 | process terminated 12:35:40 resque.1 | process terminated 12:35:40 redis.1 | process terminated

    Read the article

  • Logstash shipper & server on the samebox

    - by keftes
    I'm trying to setup a central logstash configuration. However I would like to be sending my logs through syslog-ng and not third party shippers. This means that my logstash server is accepting via syslog-ng all the logs from the agents. I then need to install a logstash process that will be reading from /var/log/syslog-clients/* and grabbing all the log files that are sent to the central log server. These logs will then be sent to redis on the same VM. In theory I need to also configure a second logstash process that will read from redis and start indexing the logs and send them to elasticsearch. My question: Do I have to use two different logstash processes (shipper & server) even if I am in the same box (I want one log server instance)? Is there any way to just have one logstash configuration and have the process read from syslog-ng --- write to redis and also read from redis --- output to elastic search ? Diagram of my setup: [client]-------syslog-ng--- [log server] ---syslog-ng <----logstash-shipper --- redis <----logstash-server ---- elastic-search <--- kibana

    Read the article

  • Architectural advice - web camera remote access

    - by Alan Hollis
    I'm looking for architectural advice. I have a client who I've built a website for which essentially allows users to view their web cameras remotely. The current flow of data is as follows: User opens page to view web camera image. Javascript script polls url on server ( appended with unique timestamp ) every 1000ms Ftp connection is enabled for the cameras ftp user. Web camera opens ftp connection to server. Web camera begins taking photos. Web camera sends photo to ftp server. On image url request: Server reads latest image on hard drive uploaded via ftp for camera. Server deleted any older images from the server. This is working okay at the moment for a small amount of users/cameras ( about 10 users and around the same amount of cameras), but we're starting to worrying about the scalability of this approach. My original plan was instead of having the files read from the server, the web server would open up an ftp connection to the web server and read the latest images directly from there meaning we should have been able to scale horizontally fairly easily. But ftp connection establishment times were too slow ( mainly due to the fact that PHP out of the ox is unable to persist ftp connections ) and so we abandoned this approach and went straight for reading from the hard drive. The firmware provider for the cameras state they're able to build a http client which instead of using ftp to upload the image could post the image to a web server. This seems plausible enough to me, but I'm looking for some architectural advice. My current thought is a simple Nginx/PHP/Redis stack. Web camera issues post requests of latest image to Nginx/PHP and the latest image for that camera is stored in Redis. The clients can then pull the latest image from Redis which should be extremely quick as the images will always be stored in memory. The data flow would then become: User opens page to view web camera image. Javascript script polls url on server ( appended with unique timestamp ) every 1000ms Camera is sent an http request to start posting images to a provided url Web camera begins taking photos. Web camera sends post requests to server as fast as it can On image url request: Server reads latest image from redis Server tells redis to delete later image My questions are: Are there any greater overheads of transferring images via HTTP instead of FTP? Is there a simple way to calculate how many potential cameras we could have streaming at once? Is there any way to prevent potentially DOS'ing our own servers due to web camera requests? Is Redis a good solution to this problem? Should I abandon PHP/Ngix combination and go for something else? Is this proposed solution actually any good? Will adding HTTPs to the mix cause posting the image to become too slow? Thanks in advance Alan

    Read the article

  • How to install stuff on Ubuntu

    - by Industrial
    Hi everyone, I have just launched my first EC2 instance and choosed a Ubuntu image to start from, since it's quite well documented. However, I am trying to install the Redis package: http://packages.ubuntu.com/lucid/redis-server Maybe I am not googling properly or just stupid since the weekend is approaching, but I'll keep getting errors: root@ip-10-229-123-199:~# sudo apt-get install redis-server Reading package lists... Done Building dependency tree... Done E: Couldn't find package redis-server I'll assume that I need to add a repository or something to Ubuntu to help it find the package I want, but how do I do it? I can only find graphical guides which doesnt help me too much since I am using SSH. Thanks alot!

    Read the article

  • Installing Windows Platform SDK Problem [on hold]

    - by user1879097
    I cannot seem to install the windows platform sdk when i have visual studio 2010 installed,i followed the error code the sdk was getting and it said i need to unistall the 2010 redistribute runtimes,i did that and it has still not fixed the problem,this is very anoying as i have tried different things and been at it for atleast 5 hours now,did anyone else get this problem and know a work around? This is the order i tried install vs 2010, remove redis runtimes, install platform sdk (failed), install redis x86/x64, install service pack 1 for vs Thanks

    Read the article

  • Architecture for dashboard showing aggregated stats [on hold]

    - by soulnafein
    I'd like to know what are common architectural pattern for the following problem. Web application A has information on sales, users, responsiveness score, etc. Some of this information are computationally intensive and or have a complex business logic (e.g. responsiveness score). I'm building a separate application (B) for internal admin tasks that modifies data in web application A and report on data from web application A. For writing I'm planning to use a restful api. E.g. create a new entity, update entity, etc. In application B I'd like to show some graphs and other aggregate data for the previous 12 months. I'm planning to store the aggregate data for each month in redis. Some data should update more often, e.g every 10 minutes. I can think of 3 ways of doing this. A scheduled task in app B that connects to an api of app A that provides some aggregated data. Then app B stores it in Redis and use that to visualise pages. Cons: it makes complex calculation within a web request, requires lot's of work e.g. api server and client, storing, etc., pros: business logic still lives in app A. A scheduled task in app A that aggregates data in an non-web process and stores it directly in Redis to be accessed by app B. A scheduled task in app A that aggregates data in a non-web process and uses an api in app B to save it. I'd like to know if there is a well known architectural solution to this type of problems and if not what are other pros/cons for the solution I've suggested?

    Read the article

  • What "pieces" are needed in order to set up a cluster of physical servers?

    - by Chris Dutrow
    Background: Currently, we use Rackspace cloud servers. We have no intention to stop using them, but would like to look into setting up a cluster of physical servers (probably desktop computers in the $400 range with 8gb memory each) to offset some of our load and work as a secondary, more powerful, less reliable system. To put things in perspective, we can buy comparable desktop computers for the same price as we pay in one month to rent them on Rackspace Cloud. I understand that this is generally a dumb idea. However, in this particular instance, the server cluster is needed for its computation power. It is not mission-critical, it does not host a consumer-facing website, and if it goes down for a day or two, its not really a problem. Currently, we have access to business class verizon fios. If I understand correctly, we can get at least 25 dedicated IP addresses with this service, this should be enough. Requirements: Each server runs Linux Centos 6.3 Some of the servers run Python and execute processes from a task queue (Redis or RabbitMQ) Some of the servers are capable of serving static files and Python driven REST APIs Some of the servers host a Cassandra database cluster One or more of the servers are a Redis database servers One or more of the servers are PostgreSQL servers Questions: What kind of router or switch is needed? We would like the computers to be able to communicate effectively with each other via internal IP addresses. This is especially important for communicating with servers hosting Redis that need to be able to respond to requests very quickly. Are there special switches or routers that need to be used to connect the servers together? Are Desktop computers ok for this? We have found that we are mostly RAM-bottle necked, I understand that some servers have highly superior CPUs, but I'm not sure we need CPU power as much as we need RAM, which is cheap in Desktop computers. Will we have problems with the WIFI cards in the desktops or any other unexpected hardware limitation? What tools should be used to "image" the servers. For example, when we get an installation right for a Redis server or Cassandra node, are there tools that come with Linux Centos 6.3 to image the server to a USB drive or something like that? Or do we need to use some other software for this? What other things are we missing that we should be concerned about? Thanks so much!

    Read the article

  • node.js storing gamestate, how?

    - by expressnoob
    I'm writing a game in javascript, and to prevent cheating, i'm having the game be played on the server (it's a board game like a more complicated checkers). Since the game is fairly complex, I need to store the gamestate in order to validate client actions. Is it possible to store the gamestate in memory? Is that smart? Should I do that? If so, how? I don't know how that would work. I can also store in redis. And that sort of thing is pretty familiar to me and requires no explanation. But if I do store in redis, the problem is that on every single move, the game would need to get the data from redis and interpret and parse that data in order to recreate the gamestate from scratch. But since moves happen very frequently this seems very stupid to me. What should I do?

    Read the article

  • Roanoke Code Camp 2014

    - by Brian Lanham
    Originally posted on: http://geekswithblogs.net/codesailor/archive/2014/05/18/156407.aspxI had a great time yesterday at Roanoke Code Camp!  Many thanks to American National University for the venue, the code camp staff and volunteers, the other speakers, and of course the attendees who made my sessions interactive.  I learned a lot yesterday and it was a good time all around. I attended sessions on Apache Cassandra by Dr. Dave King (@tildedave), Angular JS by Kevin Israel (@kevadev), and JavaScript for Object-Oriented Programmers by Joel Cochran (@joelcochran).  I regret I was unable to attend all the sessions. I also had the opportunity to present.  I spoke on Redis and got some people excited about graph databases by talking about Neo4j.  You can find my slides and other materials at the following links: My Presentation Materials Folder Redis Materials – Slides     - Snippets Neo4j Materials – Slides     - Snippets If you have any trouble getting any of the materials just respond to this post or tweet me @codesailor and I will make sure you get the information you need.

    Read the article

  • Saving all hits to a web app

    - by bevanb
    Are there standard approaches to persisting data for every hit that a web app receives? This would be for analytics purposes (as a better alternative to log mining down the road). Seems like Redis would be a must. Is it advisable to also use a different DB server for that table, or would Redis be enough to mitigate the impact on the main DB? Also, how common is this practice? Seems like a no brainer for businesses who want to better understand their users, but I haven't read much about it.

    Read the article

  • GitLab post-receive hook not firing

    - by Ben Graham
    Apologies if this isn't the right stackexchange. I have a GitLab install. It was installed over the top of a gitolite install that was only a few days old, and I assume this non-standard setup is at the root of my problem, but I cannot pin it down. The problem is straightforward: post-receive hooks are not fired. This prevents 'project activity' appearing in GitLab. The problem looks like: $ git push #... error: cannot run hooks/post-receive: No such file or directory Hook Exists The post-receive hook/symlink exists and is executable: -rwxr-xr-x 1 git git 470 Oct 3 2012 .gitolite/hooks/common/post-receive lrwxrwxrwx 1 git git 45 Oct 3 2012 repositories/project.git/hooks/post-receive -> /home/git/.gitolite/hooks/common/post-receive It's Executable By GitLab The gitlab user can execute the script (I have removed the /dev/null redirect and fed in blank input to get an 'OK' as output): sudo su - gitlab -c /home/git/.gitolite/hooks/common/post-receive OK GitLab Can Find It GitLab is looking for hooks in the correct location: $ grep hooks /srv/gitlab/gitlab/config/gitlab.yml hooks_path: /home/git/.gitolite/hooks/ and $ bundle exec rake gitlab:app:status RAILS_ENV=production # ... /home/git/.gitolite/hooks/common/post-receive exists? ............YES Environment The env -i line in the hook is commonly cited as an issue. I think that would occur after this problem, but for completeness, redis-cli is found OK: $ env -i redis-cli redis> I've run out of debugging ideas on this one. Does anybody have any suggestions?

    Read the article

  • casting doubles to integers in order to gain speed

    - by antirez
    Hello all, in Redis (http://code.google.com/p/redis) there are scores associated to elements, in order to take this elements sorted. This scores are doubles, even if many users actually sort by integers (for instance unix times). When the database is saved we need to write this doubles ok disk. This is what is used currently: snprintf((char*)buf+1,sizeof(buf)-1,"%.17g",val); Additionally infinity and not-a-number conditions are checked in order to also represent this in the final database file. Unfortunately converting a double into the string representation is pretty slow. While we have a function in Redis that converts an integer into a string representation in a much faster way. So my idea was to check if a double could be casted into an integer without lost of data, and then using the function to turn the integer into a string if this is true. For this to provide a good speedup of course the test for integer "equivalence" must be fast. So I used a trick that is probably undefined behavior but that worked very well in practice. Something like that: double x = ... some value ... if (x == (double)((long long)x)) use_the_fast_integer_function((long long)x); else use_the_slow_snprintf(x); In my reasoning the double casting above converts the double into a long, and then back into an integer. If the range fits, and there is no decimal part, the number will survive the conversion and will be exactly the same as the initial number. As I wanted to make sure this will not break things in some system, I joined #c on freenode and I got a lot of insults ;) So I'm now trying here. Is there a standard way to do what I'm trying to do without going outside ANSI C? Otherwise, is the above code supposed to work in all the Posix systems that currently Redis targets? That is, archs where Linux / Mac OS X / *BSD / Solaris are running nowaday? What I can add in order to make the code saner is an explicit check for the range of the double before trying the cast at all. Thank you for any help.

    Read the article

  • How to rate-limit concurrent sessions with nginx or haproxy?

    - by bantic
    I'm currently using nginx to reverse-proxy requests from web clients that are doing long-polling to an upstream. Since we're doing long polling (as opposed to websockets), when a client connects it will make multiple http connections to the server in serial, re-establishing a connection every time the server sends it some data (or timing out and re-establishing if the server has nothing to say for 10 seconds). What I'd like to do is limit the number of concurrent web clients. Since the clients are constantly making new HTTP requests instead of keeping a single request open, it's a little tricky to count the total number of web clients (because it's not the same as total number of concurrently connected http clients). The method I've come up with is to track http requests by the originating IP address, and store the IP address somewhere with a TTL of 20 seconds. If a request comes in whose IP isn't recognized, then we check the total number of unexpired stored IP addresses; if that's less than the maximum then we allow this request through. And if a request comes in with an IP address that we can find in the look-up table that hasn't yet expired, then it is allowed through as well. All requests that are allowed through have their IPs added to the table (if not there before) and the TTL refreshed to 20 seconds again. I had actually whipped something together that worked correctly this way using nginx along with the Redis 2.0 Nginx Module (and the nginx lua module to simplify the conditional branching), using redis to store my IP addresses with a TTL (the SETEX command), and checking the table size with the DBSIZE command. This worked but the performance was horrible. nginx and redis ended up using lots of cpu and the machine could only handle a very small number of concurrent requests. The new stick-table and tracking counters that were added to Haproxy in version 1.5 (via a commission from serverfault) seem like they might be ideal to implement exactly this sort of rate limiting, because the stick-table can track IP addresses and automatically expire entries. However, I don't see an easy way to get a total count of the unexpired entries in the stick table, which would be necessary to know the number of connected web clients. I'm curious if anyone has any suggestions, for nginx or haproxy or even for something else not mentioned here that I haven't thought of yet.

    Read the article

  • Appropriate Network switch for small server cluster

    - by Chris Dutrow
    Need to build a small business server cluster for the purpose of crunching data. It will not host a web site that needs to be available 24/7. It does need to support servers that host Redis, a Cassandra database cluster, and a Python web server. Operating system will most likely be Centos 6.4 Other servers in the cluster should be able to communicate very fast with each other, especially the Redis server. This will probably require the use of internal IP addresses. We will need to use multi-data center replication to synchronize the Cassandra cluster with the one that we currently have hosted on the cloud Was looking into network switches and we are unsure of the appropriate specifications that we should be looking for. Does the switch need to be "managed" or can it be "unmanged"? Does the switch need to support IPv6 or just IPv4? Do we need an enterprise level Cisco switch, or can we go with something like a $200 DLink managed (or unmanaged) small business switch? Thanks so much!

    Read the article

  • LSB Script: how do i know if something goes wrong?

    - by ianaz
    How do I know if a LSB script fails to load or where do I check the log of the lsbs scripts? I added two scripts with the following command: update-rc.d scriptname defaults And just one launches the things I need. It does not seem to be a script error since if I launch it with /etc/init.d/scriptname it works. This is my script: #!/bin/bash ### BEGIN INIT INFO # Provides: nodes # Required-Start: $remote_fs $syslog # Required-Stop: $remote_fs $syslog # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Starts all node apps # Description: Starts all node apps like AAM, AMT,... ### END INIT INFO echo "Launch Node applications with forever" export PATH=/usr/local/bin:$PATH # Starts the redis server redis-server # Starts AAM forever -o /var/log/AAM.log -e /var/log/AAM.log --spinSleepTime 2000 -m 5 start /var/nodejs/AAM/app.js

    Read the article

  • Java - System design with distributed Queues and Locks

    - by sunny
    Looking for inputs to evaluate a design for a system (java) which would have a distributed queue serving several (but not too many) nodes. These nodes would process objects present in the distributed queue and on occasion require a distributed lock across the cluster on an arbitrary (distributed) data structures. These (distributed) data structures could potentially lie in a distributed cache. Eliminating Terracotta (DSO),Hazelcast and Akka what could be alternative choices. Currently considering zookeeper as a distributed locking mechanism. Since the recommendation of a znode is not to exceed the 1M size , the understanding is that zookeeper should not be used a distributed queue. And also from Netflix curator tech note 4. So should a distributed cache, say like memcached, or redis be used to emulate a distributed queue ? i.e. The distributed queue will be stored in the caches and will be locked cluster-wide via zookeeper. Are there potential pitfalls with this high-level approach. The objects don't need to be taken off the queue. The object will pass through a lifecycle which will determine its removal from the queue. There would be about 10k+ objects in a queue at a given time changing states and any node could service one stage of the object's lifecycle. (Although not strictly necessary .. i.e. one node could serve the entire lifecycle if that is more efficient.) Any suggestions/alternatives ? sidenote: new to zookeeper ; redis etc.

    Read the article

  • Rails/Node.js interaction

    - by lpvn
    I and my co-worker are developing a web application with rails and node.js and we can't reach a consensus regarding a particular architectural decision. Our setup is basically a rails server working with node.js and redis, when a client makes a http request to our rails API in some cases our rails application posts the response to a redis database and then node.js transmits the response via websocket. Our disagreement occurs in the following point: my co-worker thinks that using node.js to send data to clients is somewhat business logic and should be inside the model, so in the first code he wrote he used commands of broadcast in callbacks and other places of the model, he's convinced that the models are the best place for the interaction between rails and node. I on the other hand think that using node.js belongs to the runtime realm, my take is that the broadcast commands and other node.js interactions should be in the controller and should only be used in a model if passed through a well defined interface, just like the situation when a model needs to access the current user of a session. At this point we're tired of arguing over this same thing and our discussion consists in us repeating to ourselves our same opinions over and over. Could anyone, preferably with experience in the same setup, give us an unambiguous response saying which solution is more adequate and why it is?

    Read the article

  • Distributed transactions and queues, ruby, erlang

    - by chrispanda
    I have a problem that involves several machines, message queues, and transactions. So for example a user clicks on a web page, the click sends a message to another machine which adds a payment to the user's account. There may be many thousands of clicks per second. All aspects of the transaction should be fault tolerant. I've never had to deal with anything like this before, but a bit of reading suggests this is a well known problem. So to my questions. Am I correct in assuming that secure way of doing this is with a two phase commit, but the protocol is blocking and so I won't get the required performance? It appears that DBs like redis and message queuing system like Rescue, RabbitMQ etc don't really help me a lot - even if I implement some sort of two phase commit, the data will be lost if redis crashes because it is essentially memory-only. All of this has led me to look at erlang - but before I wade in and start learning a new language, I would really like to understand better if this is worth the effort. Specifically, am I right in thinking that because of its parallel processing capabilities, erlang is a better choice for implementing a blocking protocol like two phase commit, or am I confused?

    Read the article

  • Facebook user_id as MongoDB BSON ObjectId?

    - by MattDiPasquale
    I'm rebuilding Lovers on Facebook with Sinatra & Redis. I like Redis because it doesn't have the long (12-byte) BSON ObjectIds and I am storing sets of Facebook user_ids for each user. The sets are requests_sent, requests_received, & relationships, and they all contain Facebook user ids. I'm thinking of switching to MongoDB because I want to use it's geospatial indexing. If I do, I'd want to use the FB user ids as the _id field because I want the sets to be small and I want the JSON responses to be small. But, is the BSON ObjectId better (more efficient for MongoDB) to use than just an integer (fb user_id)?

    Read the article

< Previous Page | 1 2 3 4 5 6 7 8  | Next Page >