Search Results

Search found 549 results on 22 pages for 'nosql'.

Page 9/22 | < Previous Page | 5 6 7 8 9 10 11 12 13 14 15 16  | Next Page >

  • Simplest distributed persistent key/value store that supports primary key range queries

    - by StaxMan
    I am looking for a properly distributed (i.e. not just sharded) and persisted (not bounded by available memory on single node, or cluster of nodes) key/value ("nosql") store that does support range queries by primary key. So far closest such system is Cassandra, which does above. However, it adds support for other features that are not essential for me. So while I like it (and will consider using it of course), I am trying to figure out if there might be other mature projects that implement what I need. Specifically, for me the only aspect of value I need is to access it as a blob. For key, however, I need range queries (as in, access values ordered, limited by start and/or end values). While values can have structures, there is no need to use that structure for anything on server side (can do client-side data binding, flexible value/content types etc). For added bonus, Cassandra style storage (journaled, all sequential writes) seems quite optimal for my use case. To help filter out answers, I have investigated some alternatives within general domain like: Voldemort (key/value, but no ordering) and CouchDB (just sharded, more batch-oriented); and am aware of systems that are not quite distributed while otherwise qualifying (bdb variants, tokyo cabinet itself (not sure if Tyrant might qualify), redis (in-memory store only)).

    Read the article

  • Looking for: nosql (redis/mongodb) based event logging for Django

    - by Parand
    I'm looking for a flexible event logging platform to store both pre-defined (username, ip address) and non-pre-defined (can be generated as needed by any piece of code) events for Django. I'm currently doing some of this with log files, but it ends up requiring various analysis scripts and ends up in a DB anyway, so I'm considering throwing it immediately into a nosql store such as MongoDB or Redis. The idea is to be easily able to query, for example, which ip address the user most commonly comes from, whether the user has ever performed some action, lookup the outcome for a specific event, etc. Is there something that already does this? If not, I'm thinking of this: The "event" is a dictionary attached to the request object. Middleware fills in various pieces (username, ip, sql timing), code fills in the rest as needed. After the request is served a post-request hook drops the event into mongodb/redis, normalizing various fields (eg. incrementing the username:ip address counter) and dropping the rest in as is. Words of wisdom / pointers to code that does some/all of this would be appreciated.

    Read the article

  • Searches (and general querying) with HBase and/or Cassandra (best practices?)

    - by alexeypro
    I have User model object with quite few fields (properties, if you wish) in it. Say "firstname", "lastname", "city" and "year-of-birth". Each user also gets "unique id". I want to be able to search by them. How do I do that properly? How to do that at all? My understanding (will work for pretty much any key-value storage -- first goes key, then value) u:123456789 = serialized_json_object ("u" as a simple prefix for user's keys, 123456789 is "unique id"). Now, thinking that I want to be able to search by firstname and lastname, I can save in: f:Steve = u:384734807,u:2398248764,u:23276263 f:Alex = u:12324355,u:121324334 so key is "f" - which is prefix for firstnames, and "Steve" is actual firstname. For "u:Steve" we save as value all user id's who are "Steve's". That makes every search very-very easy. Querying by few fields (properties) -- say by firstname (i.e. "Steve") and lastname (i.e. "l:Anything") is still easy - first get list of user ids from "f:Steve", then list from "l:Anything", find crossing user ids, an here you go. Problems (and there are quite a few): Saving, updating, deleting user is a pain. It has to be atomic and consistent operation. Also, if we have size of value limited to some value - then we are in (potential) trouble. And really not of an answer here. Only zipping the list of user ids? Not too cool, though. What id we want to add new field to search by. Eventually. Say by "city". We certainly can do the same way "c:Los Angeles" = ..., "c:Chicago" = ..., but if we didn't foresee all those "search choices" from the very beginning, then we will have to be able to create some night job or something to go by all existing User records and update those "c:CITY" for them... Quite a big job! Problems with locking. User "u:123" updates his name "Alex", and user "u:456" updates his name "Alex". They both have to update "f:Alex" with their id's. That means either we get into overwriting problem, or one update will wait for another (and imaging if there are many of them?!). What's the best way of doing that? Keeping in mind that I want to search by many fields? P.S. Please, the question is about HBase/Cassandra/NoSQL/Key-Value storages. Please please - no advices to use MySQL and "read about" SELECTs; and worry about scaling problems "later". There is a reason why I asked MY question exactly the way I did. :-)

    Read the article

  • Big Data – Buzz Words: Importance of Relational Database in Big Data World – Day 9 of 21

    - by Pinal Dave
    In yesterday’s blog post we learned what is HDFS. In this article we will take a quick look at the importance of the Relational Database in Big Data world. A Big Question? Here are a few questions I often received since the beginning of the Big Data Series - Does the relational database have no space in the story of the Big Data? Does relational database is no longer relevant as Big Data is evolving? Is relational database not capable to handle Big Data? Is it true that one no longer has to learn about relational data if Big Data is the final destination? Well, every single time when I hear that one person wants to learn about Big Data and is no longer interested in learning about relational database, I find it as a bit far stretched. I am not here to give ambiguous answers of It Depends. I am personally very clear that one who is aspiring to become Big Data Scientist or Big Data Expert they should learn about relational database. NoSQL Movement The reason for the NoSQL Movement in recent time was because of the two important advantages of the NoSQL databases. Performance Flexible Schema In personal experience I have found that when I use NoSQL I have found both of the above listed advantages when I use NoSQL database. There are instances when I found relational database too much restrictive when my data is unstructured as well as they have in the datatype which my Relational Database does not support. It is the same case when I have found that NoSQL solution performing much better than relational databases. I must say that I am a big fan of NoSQL solutions in the recent times but I have also seen occasions and situations where relational database is still perfect fit even though the database is growing increasingly as well have all the symptoms of the big data. Situations in Relational Database Outperforms Adhoc reporting is the one of the most common scenarios where NoSQL is does not have optimal solution. For example reporting queries often needs to aggregate based on the columns which are not indexed as well are built while the report is running, in this kind of scenario NoSQL databases (document database stores, distributed key value stores) database often does not perform well. In the case of the ad-hoc reporting I have often found it is much easier to work with relational databases. SQL is the most popular computer language of all the time. I have been using it for almost over 10 years and I feel that I will be using it for a long time in future. There are plenty of the tools, connectors and awareness of the SQL language in the industry. Pretty much every programming language has a written drivers for the SQL language and most of the developers have learned this language during their school/college time. In many cases, writing query based on SQL is much easier than writing queries in NoSQL supported languages. I believe this is the current situation but in the future this situation can reverse when No SQL query languages are equally popular. ACID (Atomicity Consistency Isolation Durability) – Not all the NoSQL solutions offers ACID compliant language. There are always situations (for example banking transactions, eCommerce shopping carts etc.) where if there is no ACID the operations can be invalid as well database integrity can be at risk. Even though the data volume indeed qualify as a Big Data there are always operations in the application which absolutely needs ACID compliance matured language. The Mixed Bag I have often heard argument that all the big social media sites now a days have moved away from Relational Database. Actually this is not entirely true. While researching about Big Data and Relational Database, I have found that many of the popular social media sites uses Big Data solutions along with Relational Database. Many are using relational databases to deliver the results to end user on the run time and many still uses a relational database as their major backbone. Here are a few examples: Facebook uses MySQL to display the timeline. (Reference Link) Twitter uses MySQL. (Reference Link) Tumblr uses Sharded MySQL (Reference Link) Wikipedia uses MySQL for data storage. (Reference Link) There are many for prominent organizations which are running large scale applications uses relational database along with various Big Data frameworks to satisfy their various business needs. Summary I believe that RDBMS is like a vanilla ice cream. Everybody loves it and everybody has it. NoSQL and other solutions are like chocolate ice cream or custom ice cream – there is a huge base which loves them and wants them but not every ice cream maker can make it just right  for everyone’s taste. No matter how fancy an ice cream store is there is always plain vanilla ice cream available there. Just like the same, there are always cases and situations in the Big Data’s story where traditional relational database is the part of the whole story. In the real world scenarios there will be always the case when there will be need of the relational database concepts and its ideology. It is extremely important to accept relational database as one of the key components of the Big Data instead of treating it as a substandard technology. Ray of Hope – NewSQL In this module we discussed that there are places where we need ACID compliance from our Big Data application and NoSQL will not support that out of box. There is a new termed coined for the application/tool which supports most of the properties of the traditional RDBMS and supports Big Data infrastructure – NewSQL. Tomorrow In tomorrow’s blog post we will discuss about NewSQL. Reference: Pinal Dave (http://blog.sqlauthority.com) Filed under: Big Data, PostADay, SQL, SQL Authority, SQL Query, SQL Server, SQL Tips and Tricks, T SQL

    Read the article

  • Google migre AdWords de MySQL vers F1, son nouveau SGBDR distribué, développé en interne qui combine le meilleur de NoSQL et SQL

    Google migre AdWords de MySQL vers F1 son nouveau SGBDR distribué développé en interne qui combine le meilleur de NoSQL et SQL Google a développé en catimini son propre gestionnaire de base de données relationnelle. La société a déplacé récemment plusieurs de ses services de publicité de MySQL vers F1, un nouveau SGDBR « Fault-Tolerant Distributed » développé en interne. Présenté lors du forum SIGMOD 2012 de Scottsdale en Arizona sur les bases de données, F1 combine les meilleures approches des SGBDR et des bases de données NoSQL, selon la division Google Research, à l'origine du projet. F1 est essentiellement centré autour de l'évolutivité, la tolérance aux pann...

    Read the article

  • Write-only collections in MongoDB

    - by rcoder
    I'm currently using MongoDB to record application logs, and while I'm quite happy with both the performance and with being able to dump arbitrary structured data into log records, I'm troubled by the mutability of log records once stored. In a traditional database, I would structure the grants for my log tables such that the application user had INSERT and SELECT privileges, but not UPDATE or DELETE. Similarly, in CouchDB, I could write a update validator function that rejected all attempts to modify an existing document. However, I've been unable to find a way to restrict operations on a MongoDB database or collection beyond the three access levels (no access, read-only, "god mode") documented in the security topic on the MongoDB wiki. Has anyone else deployed MongoDB as a document store in a setting where immutability (or at least change tracking) for documents was a requirement? What tricks or techniques did you use to ensure that poorly-written or malicious application code could not modify or destroy existing log records? Do I need to wrap my MongoDB logging in a service layer that enforces the write-only policy, or can I use some combination of configuration, query hacking, and replication to ensure a consistent, audit-able record is maintained?

    Read the article

  • Cassandra inserts using Net::Cassandra::Easy in Perl

    - by knorv
    When using the Perl module Net::Cassandra::Easy to interface with Cassandra I use the following code to read colums col[123] from rows row[123] in column-family Standard1: my $cassandra = Net::Cassandra::Easy->new(keyspace => 'Keyspace1', server => 'localhost'); $cassandra->connect(); my $result = $cassandra->get(['row1', 'row2', 'row3'], family => 'Standard1', byname => ['col1', 'col2', 'col3']); This works as expected. However, when trying to insert row row1 with .. $result = $cassandra->mutate(['row1'], family => 'Standard1', insertions => { "col1" => "Value to set." }); .. I get the error message Can't use string ("0") as a SCALAR ref while "strict refs" in use at .../Net/GenThrift/Thrift/BinaryProtocol.pm line 376. What am I doing wrong?

    Read the article

  • Cross-platform embedded database/key-value store for C#

    - by Arne Claassen
    I'm looking for a fast, embeddable key/value store with cursor semantics over key collections (or a simple embeddable DB) that I can use in .NET and mono. Need it to be open-source, would prefer an MIT or Apache style license over a GPL license. Not opposed to a library that needs bindings to be written, as long as binaries are available for both windows and linux. Options considered: SQLite - has bindings and native implementation, but single-threaded and not all that fast Embedded InnoDB - no .NET bindings i can find and it's GPLv2 Berkley DB - no .NET bindings i can find Tokyo Cabinet - no .NET bindings i can find and problematic to build on windows MadCow Memory-mapped data structures - GPLv2 Is there an option better than the above that i'm missing, or bindings for the above i don't know about?

    Read the article

  • Complex Query on cassandra

    - by Sadiqur Rahman
    I have heard on cassandra database engine few days ago and searching for a good documentation on it. after studying on cassandra I got cassandra is more scalable than other data engine. I also read on Amazon SimpleDB but as SimpleDB has a limitation 10GB/table and Google Datastore is slower than Amazon SimpleDB, I prefer not to use them (Google Datastore, Amazon SimpleDB). So for making our site scaled specially high write rates with massive data, I like to use Cassandra as our Data Engine. But before starting using cassandra I am confused on "How to handle complex data using casssandra". I am giving you the MySQL database structure below, Please read this and give me a good suggestion. Users Table hasColum ID Primary hasColum email Unique hasColum FirstName hasColum LastName Category Table hasColum ID Primary hasColum Parent hasColum Category Posts Table hasColum ID Primary hasColum UID Index foreign key linked to users-ID hasColum CID Index foreign key linked to Category-ID hasColum Title hasColum Post Index hasColum PunDate Comments hasColum ID primary hasColum UID Index foreign key linked to users-ID hasColum PID Index foreign key linked to Posts-ID hasColum Comment User Group hasColum ID primary hasColum Name UserToGroup Table (for many to many relation only) hasColum UID foreign key linked to Users-ID hasColum GID foreign key linked to Group-ID Finally for your information, I like to use SimpleCassie PHP Class http://code.google.com/p/simpletools-php/ So, it will be very helpful if you can give me example using SimpleCassie

    Read the article

  • Best DataMining Database

    - by Eric
    I am an ocasional Python programer who only have worked so far with MYSQL or SQLITE databases. I am the computer person for everything in a small compamy and I have been started a new project where I think it is about time to try new databases. Sales departament makes a CSV dump every week and I need to make a small scripting application that allow people form other departaments mixing the information, mostly linking the records. I have all this solved, my problem is the speed, I am using just plain text files for all this and unsurprisingly it is very slow. I thought about using mysql, but then I need installing mysql in every desktop, sqlite is easier, but it is very slow. I do not need a full relational database, just some way of play with big amounts of data in a decent time. Many thanks!

    Read the article

  • Recommend a good db4o viewer

    - by mgroves
    I'm playing around with db4o, and I have the Object Manager viewer thingy for Visual Studio. It seems okay, but not exactly on par with tools like HeidiSQL/SQL Studio/etc., not to mention that it locks the db4o file--I can't use my db4o app and Object Manager at the same time. Maybe I'm using it wrong, but regardless, I'd like to know what else is out there. What tools would you recommend for looking at and manipulating db4o files?

    Read the article

  • A catalogue of Cassandra log messages: What is the correct interpretation?

    - by knorv
    The following is a complete catalogue of all log messages generated by Cassandra 0.6 when stress-testing a Cassandra installation over an extended period of time: AntiEntropyService: Sending AEService tree for (,) to: [] CassandraDaemon: Binding thrift service to localhost/N.N.N.N:N CassandraDaemon: Cassandra starting up... ColumnFamilyStore: has reached its threshold; switching in a fresh Memtable at CommitLogContext(file='.../cassandra/commitlog/CommitLog-N.log', position=N) ColumnFamilyStore: Enqueuing flush of Memtable()@N CommitLog: Discarding obsolete commit log:CommitLogSegment(.../cassandra/commitlog/CommitLog-N.log) CommitLog: Log replay complete CommitLog: Replaying .../cassandra/commitlog/CommitLog-N.log, ... CommitLogSegment: Creating new commitlog segment .../cassandra/commitlog/CommitLog-N.log CompactionManager: Compacted to .../cassandra/data//-N-Data.db. N/N bytes for N keys. Time: Nms. CompactionManager: Compacting [org.apache.cassandra.io.SSTableReader(path='.../cassandra/data//-N-Data.db'), ...] DatabaseDescriptor: Auto DiskAccessMode determined to be mmap GCInspector: GC for ConcurrentMarkSweep: N ms, N reclaimed leaving N used; max is N GCInspector: GC for ParNew: N ms, N reclaimed leaving N used; max is N Memtable: Completed flushing .../cassandra/data//-N-Data.db Memtable: Writing Memtable()@N SSTable: Deleted .../cassandra/data//-N-Data.db SSTableDeletingReference: Deleted .../cassandra/data//-N-Data.db SSTableReader: Sampling index for .../cassandra/data//-N-Data.db StorageService: Starting up server gossip SystemTable: Saved ClusterName found: Test Cluster SystemTable: Saved ClusterName not found. Using Test Cluster SystemTable: Saved Token found: N SystemTable: Saved Token not found. Using N For each of the log messages listed - what is the correct interpretation of the log message?

    Read the article

  • Does Perl's Net::Cassandra module support UTF-8?

    - by knorv
    I've run into a really strange UTF-8 problem with Net::Cassandra::Easy (which is built upon Net::Cassandra): UTF-8 strings written to Cassandra are garbled upon retrieval. The following code shows the problem: use strict; use utf8; use warnings; use Net::Cassandra::Easy; binmode(STDOUT, ":utf8"); my $key = "some_key"; my $column = "some_column"; my $set_value = "\x{2603}"; my $cassandra = Net::Cassandra::Easy->new(keyspace => "Keyspace1", server => "localhost"); $cassandra->connect(); $cassandra->mutate([$key], family => "Standard1", insertions => { $column => $set_value }); my $result = $cassandra->get([$key], family => "Standard1", standard => 1); my $get_value = $result->{$key}->{"Standard1"}->{$column}; if ($set_value eq $get_value) { # this is the path I want. print "OK: $set_value == $get_value\n"; } else { # this is the path I get. print "ERR: $set_value != $get_value\n"; } When running the code above $set_value eq $get_value evaluates to false. What am I doing wrong?

    Read the article

  • MongoDB vs. Redis vs. Cassandra for a fast-write, temporary row storage solution

    - by Mark Bao
    Hi there, I'm building a system that tracks and verifies ad impressions and clicks. This means that there are a lot of insert commands (about 90/second average, peaking at 250) and some read operations, but the focus is on performance and making it blazing-fast. The system is currently on MongoDB, but I've been introduced to Cassandra and Redis since then. Would it be a good idea to go to one of these two solutions, rather than stay on MongoDB? Why or why not? Thank you

    Read the article

  • How to implement Object Databases in Asp.net MVC

    - by amexn
    I started my project in Asp.net MVC(c#) & SQL Server 2005.I want to implement Object Databases in my project. While searched in google i found "MongoDb" & db4o I didn't have enough knowledge in Object Databases & which one best suited for SQL Server 2005. Please suggest a good example/reference regarding Object Databases implementation in Asp.net MVC application

    Read the article

  • Child objects in MongoDB

    - by Jeremy B.
    I have been following along with Rob Conery's Linq for MongoDB and have come across a question. In the example he shows how you can easily nest a child object. For my current experiment I have the following structure. class Content { ... Profile Profile { get; set; } } class Profile { ... } This works great when looking at content items. The dilemma I'm facing now is if I want to treat the Profile as an atomic object. As it stands, it appears as if I can not query the Profile object directly but that it comes packaged with Content results. If I want it to be inclusive, but also be able to query on just Profile I feel like my first instinct would be to make Profiles a top level object and then create a foreign key like structure under the Content class to tie the two together. To me it feels like I'm falling back on RDBMS practices and that feels like I'm most likely going against the spirit of Mongo. How would you treat an object you need to act upon independently yet also want as a child object of another object?

    Read the article

  • Net::Cassandra::Easy equivalent of "SELECT * FROM ..."

    - by knorv
    When using Perl's Net::Cassandra::Easy the following code will retrieve columns col[1-3] from rows row[1-3]: $result = $cassandra->get(['row1', 'row2', 'row3'], family => 'Standard1', byname => ['col1', 'col2', 'col3'); The corresponding SQL would be: SELECT col1, col2, col3 FROM rows WHERE id IN ('row1', 'row2', 'row3'); Suppose instead that I want to retrieve all columns. In SQL terms that would be: SELECT * FROM rows WHERE id IN ('row1', 'row2', 'row3'); To get all columns I am currently using: $result = $cassandra->get(['row1', 'row2', 'row3'], family => 'Standard1', byoffset => { "count" => 1_000_000 }); This works as long as the number of columns does not exceed one million. While this works I'd assume that there is a cleaner way to do it. My question is: Is there any cleaner way to specify to Cassandra that I want to retrieve all columns for the maching rows?

    Read the article

  • Pros/Cons of document based database vs relational database

    - by damian
    I've been trying to see if I can accomplish some requirements with a document based database, in this case CouchDB. Two generic requirements: CRUD of entities with some fields which have unique index on it ecommerce web app like eBay (better description here). And I'm begining to think that a Document-based database isn't the best choice to address these requirements. Furthermore, I can´t imagine a use for a Document based database (maybe my imagination is too little). Can you explain me if I am asking pears to an elm when I try to use a Document based database for this requirements?

    Read the article

  • Cassandra/HBase or just MySQL: Potential problems doing the next thing

    - by alexeypro
    Say I have "user". It's the key. And I need to keep "user count". I am planning to have record with key "user" and value "0" to "9999+ ;-)" (as many as I'll have). What problems I will drive in if I use Cassandra, HBase or MySQL for that? Say, I have thousand of new updates to this "user" key, where I need to increment the value. Am I in trouble? Locked for writes? Any other way of doing that? Why this is done -- there will be a lot of "user"-like keys. Different other cases. But the idea is the same. Why keep it this way -- because I'll have more reads, so I can always get "counted value" very fast.

    Read the article

  • How do I insert a row with Perl's Net::Cassandra::Easy?

    - by knorv
    When using the Perl module Net::Cassandra::Easy to interface with Cassandra I use the following code to read colums col[123] from rows row[123] in column-family Standard1: my $cassandra = Net::Cassandra::Easy->new(keyspace => 'Keyspace1', server => 'localhost'); $cassandra->connect(); my $result = $cassandra->get(['row1', 'row2', 'row3'], family => 'Standard1', byname => ['col1', 'col2', 'col3']); This works as expected. However, when trying to insert row row1 with .. $result = $cassandra->mutate(['row1'], family => 'Standard1', insertions => { "col1" => "Value to set." }); .. I get the error message Can't use string ("0") as a SCALAR ref while "strict refs" in use at .../Net/GenThrift/Thrift/BinaryProtocol.pm line 376. What am I doing wrong?

    Read the article

  • Mongoid or MongoMapper?

    - by PanosJee
    I have tried MongoMapper and it is feature complete (offering almost all AR functionality) but i was not very happy with the performance when using large datasets. Has anyone compared with Mongoid? Any performance gains ?

    Read the article

  • Querying a Cassandra column family for rows that have not been updated in X days

    - by knorv
    I'm moving an existing MySQL based application over to Cassandra. So far finding the equivalent Cassandra data model has been quite easy, but I've stumbled on the following problem for which I'd appreciate some input: Consider a MySQL table holding millions of entities: CREATE TABLE entities ( id INT AUTO_INCREMENT NOT NULL, entity_information VARCHAR(...), entity_last_updated DATETIME, PRIMARY KEY (id), KEY (entity_last_updated) ); The table is regularly queried for entities that need to be updated: SELECT id FROM entities WHERE entity_last_updated IS NULL OR entity_last_updated < DATE_ADD(NOW(), INTERVAL -7*24 HOUR) ORDER BY entity_last_updated ASC; The entities returned by this queries are then updated using the following query: UPDATE entities SET entity_information = ?, entity_last_updated = NOW() WHERE id = ?; What would be the corresponding Cassandra data model that would allow me to store the given information and effectively query the entities table for entities that need to be updated (that is: entities that have not been updated in the last seven days)?

    Read the article

  • Perform case-insensitive lookup on an Array in MongoDB?

    - by Hal
    So, I've decided to get my feet wet with MongoDB and love it so far. It seems very fast and flexible which is great. But, I'm still going through the initial learning curve and as such, I'm spending hours digging for info on the most basic things. I've search throughout the MongoDB online documentation and have spent hours Googling through pages without any mention of this. I know Mongo is still quite new (v1.x) so it explains why there isn't much information yet. I've even trying looking for books on Mongo without much luck. So yes, I've tried to RTFM with no luck, so, now I turn to you. I have an Array of various Hashtags nested in each document (ie: #apples, #oranges, #Apples, #APPLES) and I would like to perform a case-insensitive find() to access all the documents containing apples in any case. It seems that find does support some regex with /i, but I can't seem to get this working either. Anyway, I hope this is a quick answer for someone. Here's my existing call in PHP which is case sensitive: $cursor = $collection->find(array( "hashtags" => array("#".$keyword)))->sort(array('$natural' => -1))->limit(10); Help?

    Read the article

< Previous Page | 5 6 7 8 9 10 11 12 13 14 15 16  | Next Page >