Vijay Samuel's Blog

Great news for all Google Summer of Code aspirants. Drizzle has been accepted as a mentoring organization for the 4th time. Students are welcome to look at our wiki page link and find possible projects and mentors. We hangout at #drizzle on IRC Freenode. Feel free to drop by and post your queries. We have devs from across the globe and so you may get your queries solved at the earliest. If you cant find us on the IRC channel then just shoot out a mail to our mailing list Drizzle-Discuss. We have good documentation on how to get started with code contribution. We would love to see you submit patches for low hanging fruit when you try to get a fix on your project proposals. We hope to have good talent at Drizzle this year. Put on those thinking hats and get those ready. Happy hacking fellow students!!!


Its been a while since I ve blogged but I couldn’t think of a better time to resume blogging than when Drizzle was officially became associated to Software in the Public Interest. I ve been a part of Drizzle for almost a year and a half now and my passion for Drizzle seems to grow every day. It is always good to see changes that happen for good and this is one of them I guess. Now that Drizzle is a part of SPI, it has a legal entity behind it which is always good. How can you benefit from this you may ask. If you are a US tax payer, then any donation that you make will be tax deductible and all your valuable contributions will be used towards the betterment of Drizzle. The easiest way to donate is using a credit card at Click & Pledge. The SPI website lists some alternative methods such as using a cheque.Β  So, please do make your valuable contributions towards Drizzle. As always I feel proud to be a part of the Drizzle family and will continue to strive for the betterment of Drizzle. πŸ™‚ πŸ™‚ πŸ™‚

I ‘ve been doing some reading on Stored Procedures and how they are being defined and executed. These are some of the points which I think should be covered in our Stored Procedure Interface and some of my suggestions. I’m open to suggestions and criticism. According to what we had discussed in the channel we need make the stored procedure interface pluggable. So, a part of the interface will reside within drizzled and the client part of the interface will reside within the plugin itself.

I personally feel we could work on this interface on a series of 5 to 6 iterations.

1) Write grammar for our stored procedures, a lexical analyser and some parser code using flex and bison. I think we could abide to the SQL standards as much as possible from the earlier stages so that we don’t need to refactor much later on. After we write the grammar we need to test thoroughly!!! The earlier we find bugs the better.

2) Update sql_lex and sql_yacc so that the new keywords STORED and PROCEDURE are understood by our SQL grammar. Update the client code so
that we are able to use the STORED and PROCEDURE keywords. Update bison code to CREATE and DROP Stored Procedures. Use EXECUTE_SYM to execute the stored procedures.

3) We need to store our stored procedures on tables so we will have to write protobuffers for the new fragment of code that is going to enable us to store the stored procedures on the tables.

Now, after the third pass we could merge the code into trunk and _technically_ we should be able to run stored procedures that have only SQL statements. Once we get this working we should be able add the rest of the features with patches.

4) Determine a convention for denoting variables. SQL Server uses @ prefixed to names to denote that the given name is a variable. Enable stored procedures to accept input parameters. We will be needing to re write protobuffers because we need to use these variables in our tables and give special meaning to them in the future.( i.e if they are IN, OUT or INOUT). The interface will not support IN, OUT and INOUT in this pass though.

5) Add support for IN, OUT and INOUT. We need to think of a good way to prevent modification of IN variables. I do not know how to make a table
entry readonly. We also need to be able to return values in the case of OUT and INOUT variables. We could have a column that denotes if a variable is IN OUT or INOUT and based on the entry give write permissions on that variable. Just a suggestion.

6) Add SET to the stored procedures grammar. This will enable us to use local variables. The protobuffers need to be re written so that the local variables can be stored in out tables.

I need to do alot of reading on google protobuffers and brush up on flex and bison. The first three iterations are hardest according to me. I hope I made some sense in these notes.

Please do comment on any mistakes that I ‘ve made so that I could work on them. Better ways on approaching this problem are also welcome. πŸ™‚

Its almost a year since I first became a part of the Drizzle family and its been an amazing experience so far. I first came to Drizzle with an aspiration to become a Google Summer of Code intern. I did realise that dream under the guidance of my mentor Brian Aker. Google Summer of Code was awesome. I got to work onΒ  the command line options processing system alongside Brian, Monty Taylor and Jay Pipes. Summer of Code 2010 was a huge success and I did manage some good work into trunk.

Even though summer of code was over I did hang around at Drizzle and do some struct to class refactoring, a lot of bug fixes and some tamil utf-8 test cases. I feel proud to work along side such great people and learn new technologies with them as Drizzle grows. The number of friends I have made here are countless, Stewart Smith, Andrew Hutchings, David Shrewsbury, Patrick Crews, just to name a few. πŸ™‚

I recently approached Brian and Monty regarding what I could do for the coming Summer of Code and Brian was quick to respond. πŸ™‚ Stored Procedure Interface was his reply and SPI is what I am going to propose for the upcoming Summer of Code. Brian is very excited about it and so are many devs at #drizzle including myself. I will be coming up withΒ  a description regarding this project in a few days time and I ‘m open to any comments.

I almost forgot about the GA! The GA of Drizzle is coming out in a few days. I’m pretty excited to be a part of the team and I will definitely contribute more to Drizzle.. Thank you guys! I do love working at #drizzle. πŸ™‚ πŸ™‚ πŸ™‚

I guess it has been a while since I updated my blog probably because I ve been caught up in end of semester activities. During that time I got to work on a very interesting project as a part of my college curriculum. I wrote a “file system search engine” using C ++ and a lot of standard libraries from Boost and I named it Grabble.

There are a lot of file system search engines out there and many may wonder why would I waste my time on hacking on such an area when there are already a lot of good tools out there. I have noticed that some of the file system search engines do take a while to respond to queries. So it was kind of a “Need For Speed” venture.

In the end, I did come up with a search engine that is quite fast, may be even faster than the average engine out there but it does come with a huge constraint of memory requirement. I have given no regard to memory :P. There is a HUGE inverted index inside the heart of the search engines server which is used to service the clients queries. Since an inverted index is used it only takes unit time to retrieve data from the server. The system follows a client-server tcp connection which I had implemented using boost::asio. There are alot of threads running in the system which were implemented using boost::thread. The whole software is built on Monty Taylor. Thank you Monty!!!

As far as performance is concerned, Grabble can retrieve data in under 10ms and report the absence of data in exactly 10ms. There are some bugs(a lot actually πŸ˜‰ ) in the system but I would say that the system is still in its infancy and could be tweaked to attain perfection. So, if you are looking for a very high speed search engine then I guess its probably safe to say that you’d rather grabble than search for you data. πŸ˜‰ If you have any queries regarding grabble you re always welcome to PM me.

I’m probably the last of all the devs at Drizzle to blog about the big Beta release but I haven’t had a lot of time on my hands these days to do it. It has been six months now since I first came to the Drizzle community to try and make it into Google Summer of Code and I ‘ve seen Drizzle go through a lot of changes. One of the changes have come from myself which is replacing my_getopt with boost:: program_options which I had done as a part of Google Summer of Code under the guidance of Brian Aker. Now, the Beta may not be a big deal to many but it is a huge deal to me as this is the first time I’m a part of something huge. πŸ™‚ πŸ™‚

Even though summer of code has ended and I am still spending time in #drizzle in order to contribute patches, utf-8 test cases and bug fixes to Drizzle. (Its difficult to leave such good people and the Drizzle code base is awesome to work on πŸ˜‰ ). I will continue to contribute to Drizzle when ever I had time and see Drizzle grow as time goes by. Cheers to all my mates at #drizzle for making it so far and my wishes to them to take Drizzle even higher.

As Google Summer of Code 2010 comes to an end my work of re factoring the commandline options and configuration file processing system using boost::program_options also is nearing the grand finale which would probably mean the complete removal of my_getopt which was used to solve the above stated purpose. Over the past couple of weeks I have been working on innodb, embedded_innodb and the kernel, the three huge patches which were left.

I had stated in my last post that the two engines were in the process of being merged into trunk and I again state in this post “the two engines are in the process of being merged”. One may ask why I has taken so long to get it into trunk, which is a very logical question to ask. The answer to that is “MEMORY LEAKS”. I had failed to free memory after using strdup() causing performance deterioration. Most people would have got a good laugh out of that. πŸ˜› :P. Some more copy/paste errors seemed to prove how idiotic I can be at times. πŸ˜‰

With all that aside Monty and I dived into the kernel with high hopes of getting the job done within the next release which is tomorrow constantly being supervised by my mentor Brian. I helped lay the basic ground work for the kernel patch. I removed the last my_long_options from the kernel, did the required operations to assign the variables their required defaults, performed limit checks and block_size corrections with almost 30 notifier() ( :O :O Lot of copy/paste involved in that process :P).

Monty is now working on making the kernel patch usable. The kernel is too complex for me to handle the entire patch by myself. I really hope that he completes the patch before the release. πŸ™‚ I ve really had a wonderful time working with such wonderful people. I ‘ve been blogging as a summer intern for the past three months. My next post will probably be as a permanent developer of Drizzle. Thank you guys! Cheers!

My Timeline

April 2018
« Jan    

I, Me and Myself

My Blog Stats

  • 6,772 hits

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 4 other followers