Vijay Samuel's Blog

Drizzle 7.2.4-alpha released

vjsamuel — Sun, 23 Sep 2012 19:30:42 +0000

The 4th release of the GreenLake series is out. Major changes include:

Clang support by Brian Aker
Sysbench Qewpie testing suite by Sharan and Patrick Crews
Numerous bug fixes by coldtobi
Dynamic plugin support by Anshu Kumar
Improved Documentation

On a side note, a proof of concept was done for Drizzle as a backend for OpenStack during this release as well.

Hello,World with libdrizzle

vjsamuel — Fri, 31 Aug 2012 10:53:34 +0000

Here is a piece of code that illustrates how to create a Drizzle object, a connection object, establish a connection, execute a query, buffer its result and do some basic operations on it. The code is pretty straight forward. DRIZZLE_RETURN_OK indicates the successful execution of a function. Make sure that you have libdrizzle installed in your machine.

#include
#include
#include

int main()
{
drizzle_st *drizzle;
drizzle_con_st *con;

if ((drizzle= drizzle_create(NULL)) == NULL)
{
printf(“Error creating drizzle object\n”);
return -1;
}
con= drizzle_con_add_tcp(drizzle, NULL, “localhost”, 4427, “root”, “password”, “db”, DRIZZLE_NON_BLOCKING);

drizzle_return_t ret= drizzle_con_connect(con);
if (ret == DRIZZLE_RETURN_OK)
{
printf(“Connection Established\n”);
}

drizzle_result_st *result;
int count= 0;

result= drizzle_query(con, result, “SHOW DATABASES”, strlen(“SHOW DATABASES”), &ret);
ret= drizzle_result_buffer(result);
printf(“%d\n”, (int)drizzle_result_row_count(result));
drizzle_con_free(con);
drizzle_free(drizzle);
return 0;
}

To compile this program, one needs to link to libdrizzle. Use the following command to compile the code file. (Assuming that the code file is connect.c and that you installed libdrizzle-1.0 headers at /usr/include)
gcc -o connect connect.c -ldrizzle -I /usr/include/libdrizzle-1.0 -L /usr/lib

Hope this helps. Happy hacking folks!

An Eventful Google Summer of Code 2012

vjsamuel — Fri, 31 Aug 2012 10:52:34 +0000

This year was my third consecutive Google Summer of Code and I’ve done all three of my GSoCs for the Drizzle community. The first time I worked on refactoring the commandline options processing system using boost::program_options. The second time I worked on a stored procedure interface for Drizzle. This year I got a very interesting project to work on. My mentor, Brian Aker, asked me if I could work on adding Drizzle backend support for OpenStack. I was very thrilled since I’ve been long trying to contribute to the OpenStack community and OpenStack is a community that I’ve admired since its infant days. So, I happily said yes. Brian charted out a plan asking me to try adding a plugin for Collectd which could be used to collect the Drizzle server’s statistics from time to time. As part of this I had to work with libdrizzle. There were not many concrete examples of libdrizzle usage. I somehow managed to write a simple “Hello,World” kind of libdrizzle based C program which I had used as a base to start working on the plugin. But, because of all the relocating and stuff that was happening in my personal life I never got to finish the plugin and it is still left in a half baked state. I will add a blog showing the basic program that I had written using libdrizzle so that people could use it as a base to build on.

I then moved on to modifying devstack to script so that it uses Drizzle as a backend instead of MySQL. Monty Taylor had already told me that all the Drizzle support is already built in as a part of SQLAlchemy and I wouldn’t have to do much. Hence I started to remove all the MySQL related code and adding its equivalent Drizzle code. Diving further in I had noticed that I would have to modify some of the SQLAlchemy library code and some of the code in Nova which I did and got working OpenStack with Drizzle. I will add another blog post covering how I had achieved the backend support as well.

All in all, I had fun working on this project this year and the amount of satisfaction that I got on seeing OpenStack run with Drizzle in the backend is priceless. Ultimately I too have contributed to OpenStack in one form. Now, Drizzle is truely “on the cloud”.

Drizzle 7.1 GA

vjsamuel — Wed, 04 Apr 2012 16:21:09 +0000

The GA of Drizzle is available for public use. You can see our blog post for the new improvements available with this release. I’ve been with Drizzle for two years now and it is good to see Drizzle mature. This release has given more importance to the users as we’ve done a lot of work on improving documentation and fixing a lot of bugs. We’ve also moved forward in the quest of bringing true multi tenancy. The release is right on time for Percona Live MySQL Conference which is scheduled for next week. This release is even more special to me as I have the privilege of being Drizzle’s Release Manager and had the opportunity to announce to the world the GA of Drizzle 7.1. As always I’m proud to be a Drizzle dev

And it’s that time of the year again! GSOC 2012

vjsamuel — Fri, 16 Mar 2012 18:53:24 +0000

Great news for all Google Summer of Code aspirants. Drizzle has been accepted as a mentoring organization for the 4th time. Students are welcome to look at our wiki page link and find possible projects and mentors. We hangout at #drizzle on IRC Freenode. Feel free to drop by and post your queries. We have devs from across the globe and so you may get your queries solved at the earliest. If you cant find us on the IRC channel then just shoot out a mail to our mailing list Drizzle-Discuss. We have good documentation on how to get started with code contribution. We would love to see you submit patches for low hanging fruit when you try to get a fix on your project proposals. We hope to have good talent at Drizzle this year. Put on those thinking hats and get those ready. Happy hacking fellow students!!!

Drizzle, now a part of Software in the Public Interest

vjsamuel — Thu, 06 Oct 2011 15:58:08 +0000

Its been a while since I ve blogged but I couldn’t think of a better time to resume blogging than when Drizzle was officially became associated to Software in the Public Interest. I ve been a part of Drizzle for almost a year and a half now and my passion for Drizzle seems to grow every day. It is always good to see changes that happen for good and this is one of them I guess. Now that Drizzle is a part of SPI, it has a legal entity behind it which is always good. How can you benefit from this you may ask. If you are a US tax payer, then any donation that you make will be tax deductible and all your valuable contributions will be used towards the betterment of Drizzle. The easiest way to donate is using a credit card at Click & Pledge. The SPI website lists some alternative methods such as using a cheque. So, please do make your valuable contributions towards Drizzle. As always I feel proud to be a part of the Drizzle family and will continue to strive for the betterment of Drizzle.

Stored Procedure Interface for Drizzle

vjsamuel — Wed, 23 Mar 2011 07:36:52 +0000

I ‘ve been doing some reading on Stored Procedures and how they are being defined and executed. These are some of the points which I think should be covered in our Stored Procedure Interface and some of my suggestions. I’m open to suggestions and criticism. According to what we had discussed in the channel we need make the stored procedure interface pluggable. So, a part of the interface will reside within drizzled and the client part of the interface will reside within the plugin itself.

I personally feel we could work on this interface on a series of 5 to 6 iterations.

1) Write grammar for our stored procedures, a lexical analyser and some parser code using flex and bison. I think we could abide to the SQL standards as much as possible from the earlier stages so that we don’t need to refactor much later on. After we write the grammar we need to test thoroughly!!! The earlier we find bugs the better.

2) Update sql_lex and sql_yacc so that the new keywords STORED and PROCEDURE are understood by our SQL grammar. Update the client code so
that we are able to use the STORED and PROCEDURE keywords. Update bison code to CREATE and DROP Stored Procedures. Use EXECUTE_SYM to execute the stored procedures.

3) We need to store our stored procedures on tables so we will have to write protobuffers for the new fragment of code that is going to enable us to store the stored procedures on the tables.

Now, after the third pass we could merge the code into trunk and _technically_ we should be able to run stored procedures that have only SQL statements. Once we get this working we should be able add the rest of the features with patches.

4) Determine a convention for denoting variables. SQL Server uses @ prefixed to names to denote that the given name is a variable. Enable stored procedures to accept input parameters. We will be needing to re write protobuffers because we need to use these variables in our tables and give special meaning to them in the future.( i.e if they are IN, OUT or INOUT). The interface will not support IN, OUT and INOUT in this pass though.

5) Add support for IN, OUT and INOUT. We need to think of a good way to prevent modification of IN variables. I do not know how to make a table
entry readonly. We also need to be able to return values in the case of OUT and INOUT variables. We could have a column that denotes if a variable is IN OUT or INOUT and based on the entry give write permissions on that variable. Just a suggestion.

6) Add SET to the stored procedures grammar. This will enable us to use local variables. The protobuffers need to be re written so that the local variables can be stored in out tables.

I need to do alot of reading on google protobuffers and brush up on flex and bison. The first three iterations are hardest according to me. I hope I made some sense in these notes.

Please do comment on any mistakes that I ‘ve made so that I could work on them. Better ways on approaching this problem are also welcome.

A year at Drizzle

vjsamuel — Sat, 12 Mar 2011 13:03:01 +0000

Its almost a year since I first became a part of the Drizzle family and its been an amazing experience so far. I first came to Drizzle with an aspiration to become a Google Summer of Code intern. I did realise that dream under the guidance of my mentor Brian Aker. Google Summer of Code was awesome. I got to work on the command line options processing system alongside Brian, Monty Taylor and Jay Pipes. Summer of Code 2010 was a huge success and I did manage some good work into trunk.

Even though summer of code was over I did hang around at Drizzle and do some struct to class refactoring, a lot of bug fixes and some tamil utf-8 test cases. I feel proud to work along side such great people and learn new technologies with them as Drizzle grows. The number of friends I have made here are countless, Stewart Smith, Andrew Hutchings, David Shrewsbury, Patrick Crews, just to name a few.

I recently approached Brian and Monty regarding what I could do for the coming Summer of Code and Brian was quick to respond. Stored Procedure Interface was his reply and SPI is what I am going to propose for the upcoming Summer of Code. Brian is very excited about it and so are many devs at #drizzle including myself. I will be coming up with a description regarding this project in a few days time and I ‘m open to any comments.

I almost forgot about the GA! The GA of Drizzle is coming out in a few days. I’m pretty excited to be a part of the team and I will definitely contribute more to Drizzle.. Thank you guys! I do love working at #drizzle.

Why search when you can “Grabble”

vjsamuel — Sat, 04 Dec 2010 18:11:32 +0000

I guess it has been a while since I updated my blog probably because I ve been caught up in end of semester activities. During that time I got to work on a very interesting project as a part of my college curriculum. I wrote a “file system search engine” using C ++ and a lot of standard libraries from Boost and I named it Grabble.

There are a lot of file system search engines out there and many may wonder why would I waste my time on hacking on such an area when there are already a lot of good tools out there. I have noticed that some of the file system search engines do take a while to respond to queries. So it was kind of a “Need For Speed” venture.

In the end, I did come up with a search engine that is quite fast, may be even faster than the average engine out there but it does come with a huge constraint of memory requirement. I have given no regard to memory . There is a HUGE inverted index inside the heart of the search engines server which is used to service the clients queries. Since an inverted index is used it only takes unit time to retrieve data from the server. The system follows a client-server tcp connection which I had implemented using boost::asio. There are alot of threads running in the system which were implemented using boost::thread. The whole software is built on Monty Taylor. Thank you Monty!!!

As far as performance is concerned, Grabble can retrieve data in under 10ms and report the absence of data in exactly 10ms. There are some bugs(a lot actually ) in the system but I would say that the system is still in its infancy and could be tweaked to attain perfection. So, if you are looking for a very high speed search engine then I guess its probably safe to say that you’d rather grabble than search for you data. If you have any queries regarding grabble you re always welcome to PM me.

Being a part of the Beta release

vjsamuel — Wed, 06 Oct 2010 13:23:08 +0000

I’m probably the last of all the devs at Drizzle to blog about the big Beta release but I haven’t had a lot of time on my hands these days to do it. It has been six months now since I first came to the Drizzle community to try and make it into Google Summer of Code and I ‘ve seen Drizzle go through a lot of changes. One of the changes have come from myself which is replacing my_getopt with boost:: program_options which I had done as a part of Google Summer of Code under the guidance of Brian Aker. Now, the Beta may not be a big deal to many but it is a huge deal to me as this is the first time I’m a part of something huge.

Even though summer of code has ended and I am still spending time in #drizzle in order to contribute patches, utf-8 test cases and bug fixes to Drizzle. (Its difficult to leave such good people and the Drizzle code base is awesome to work on ). I will continue to contribute to Drizzle when ever I had time and see Drizzle grow as time goes by. Cheers to all my mates at #drizzle for making it so far and my wishes to them to take Drizzle even higher.