Vijay Samuel's Blog

Archive for May 2010

I ‘ve finally completed refactoring both command-line options and the configuration file processing and filed in my merge proposal. It was a long process and now that it ‘s finally done there are a lot of changes in drizzleslap. (Anyone whos currently working on slap would get a serious head ache when he re merges with trunk after my code is merged into trunk!!! 😉 🙂 )

Some of the changes that have been incorporated are as follows:

1) The struct my_long_options[] has been removed and program_options::options_description is used to define an object long_options.

2) The command line parsing and configuration file processing is taken care of by the parse_command_line() and the parse_config_file() instead of using custom code and so one won’t be able to find the handle_options any more.

3) get_options() has been renamed to process_options since it job now is just to process option values and it now only takes void as a parameter.

4) The internal::load_defaults() is no longer used and is replaced by the parse_config_file from boost.

5) Drizzle will no longer use the single configuration file drizzle.cnf instead it will be using mulitple configuration files such as drizzleslap.cnf and client.cnf which will be found in ~/.drizzle and SYSCONFDIR/drizzle directories. The user configuration file over rides the system configuration file. The command line over rides both configuration files.

6) get_one_option() has been removed since it is of no use any more.

Similar changes will be implemented in the server side soon. Hopefully all the custom code used for command line and configuration file parsing will be replaced by standard boost:: program_options within 2 to 3 months. 🙂

Advertisements

This is something that my mentor Brian Aker and Monty Taylor taught me. Running manual test on slap was a bit tricky at first but now I ‘m used to it. What is most important is to start a server which can be done using the following command.

First cd into the tests directory and execute the command:
./test-run –start-and-exit

You ‘ll get something like this:

Logging: ./test-run –start-and-exit
Drizzle Version 2010.05.1543
Using MTR_BUILD_THREAD             = -69.4
Using MASTER_MYPORT                    = 9306
Using MASTER_MYPORT1                  = 9307
Using SLAVE_MYPORT                        = 9308
Using SLAVE_MYPORT1                      = 9309
Using SLAVE_MYPORT2                      = 9310
Using MC_PORT                                      = 9316
Killing Possible Leftover Processes
Removing Stale Files
Creating Directories
================================================================================
DEFAULT STORAGE ENGINE: innodb
TEST                                                         RESULT    TIME (ms)
——————————————————————————–

Then come back to the root directory of your branch and after that you can choose to either run the slap program directory or in debug mode using GDB with the following command:

./libtool –mode=execute gdb ./client/drizzleslap

Once you enter the debug mode of gdb it is important to set the required arguments using set args.

Example: set args –port=9306 –silent –auto-generate-sql

(Specifying the port number is very important)

If you type run and hit enter then the program starts executing in debug mode.

This is something which is very useful when manual debugging is required.

Well, it has been almost a couple of weeks since my last post, but I have my reasons. With the help of all my mentors at drizzle especially Monty, I ‘ve literally pole vaulted my way through several obstacles and yet I ‘m not “Scot free” ( I initially had the intentions of posting only after my first branch but it turns out that it ain ‘t gonna happen for some more time 😦 ).

Turns out that CentOS and FreeBSD had older versions of boost(1.33) on the other hand I was coding with boost::program_options 1.39. This meant that I could not use some of the newer functions like implicit_value(). Monty forced in the upgrade of boost in every machine which solved this issue. Thanks a tonne Monty!!!

Another issue that bugged me a lot while coding my first branch was the older compiler version in CentOS(and unfortunately, it still does). But now, CentOS has upgraded to the gcc44 compiler. 🙂 🙂 (happy thoughts!!).

The last and worst of my issues at hand which is yet to be solved by the is that we are not sure as to how to make several programs share the same configuration file but access it’s own section. The documentation of configuration file processing is a but weakly documented. 😦 So this is the issue at hand which is yet to be solved. If we get through this then my first refactor branch will follow shortly.

Looking deeper and deeper into the drizzleslap.cc and learning boost::program_options I was able to come up with more and more solutions to remove the custom code from slap. I have completely removed my_long_options[] from slap(which will be a pleasing sight to most of the people working on drizzle after the branch gets merged I guess 😉 ). The changes that I intend to bring into slap within the next couple of days are as follows:

1. Bring program_options::options_description into main() instead of having it global(my initial assumption was that po::options_description would be used by several functions. I proved to myself that this assumption is wrong. 🙂 ) .

2. Refactor the get_options() function to accept variable_map from boost::program_options as a parameter instead of the command line and query the variable_map for the values of **value. This way, the handle_options() can be safely removed from slap.

It is important that the get_options() is called from main() only after the variable_map is built and the command line has been parsed, otherwise it would make no sense.

If all this is right then I’ ll be able to refactor slap completely in the next 1 and a half days and submit my branch. 🙂

This is all that I’ve come up with so far. Now, initially we have to replace the my_long_options with program_options::options_description. So I propose to completely remove the my_long_options[] and place the program_options::options_description globally so that several functions may call upon it as and when necessary.

Values of the options in the program have to be stored in program_options::options_description as follows.

Data member in struct option        Value Syntax/Semantic in                        Option_Description

name                    option name(straight forward)
id                          short version(again straight forward)
comment             description(again straight forward)
def_value            default_value()
var_type              value<var_type>()

arg_type              use zero_tokens(),multi_tokens()

**value    and **u_max_value        value<var_type>(address of value)

I couldn’t come up with solutions for min_value,max_value,less_value and block_size. Instead of having the program describe all the attributes of an option I propose to use a central configuration file that takes care of this so that the size of the program_options::options_description is minimum. If this is feasible then we could just have the above stated data members in the options_description and store the rest in the configuration file.

Apart from all this I ran some basic ack-grep searches for some of the usage of the data members of struct option and I think you would find the results quite interesting on some of the members.
Issue no 1.
ack-grep app_type

drizzled/option.h
70:  void       *app_type;                 /* To be used by an application */

//This is the declaration fo app_type in option.h

drizzled/plugin/loader.cc
1601:    options->app_type= opt;

Only one place uses the data member app_type and all definitions of the struct option set the value of app_type to the value 0 so I propose to scrap the data member app_type and find an aternative to replace the line “options->app_type=opt;”.

Issue no 2.
I ran an ack-grep search on option.h to see how many files depend on option.h and I got this:
ack-grep option.h

client/client_priv.h
plugin/archive/archive_performance.cc
plugin/archive/concurrency_test.cc
plugin/archive/archive_reader.cc
plugin/archive/archive_test.cc
plugin/myisam/mi_check.cc
plugin/myisam/ha_myisam.cc
drizzled/option.cc
drizzled/set_var.cc
drizzled/plugin/loader.cc
drizzled/drizzled.cc
drizzled/parser.h
drizzled/statement/set_option.cc
drizzled/internal/default.cc
Makefile
tests/resolve_stack_dump.cc
gnulib/getopt.c
extra/my_print_defaults.cc

After that I went into each file and did searches on key words like “option”,”struct option” and strangely in certain files I couldn’t find any. The code files were independent of the file “option.h” yet they seem to be have included. I tried commenting out the include statements and re running the test and the tests ran perfectly. The files which are independent are stated below:

i)plugin/archive/archive_performance.cc
ii)plugin/archive/archive_test.cc
iii)plugin/myisam/mi_check.cc
iv)plugin/archive/concurrency_test.cc

If you think the above issue has been well explained we could remove the includes from those files permanently to avoid confusion for developers who use the code at later stages.

When I first came into drizzle all I knew was “Write code, run make, then make test and push code”. If all went well then life is all happy with flowers and sunshine. 🙂 But yesterday I hit a brick wall called param-build which made me realize that it is not enough that code is error free in my machine but also error free in almost all linux-based operating systems but 32 bit and amd64. So, I had to make my code platform independent in order for it to be merged into trunk. 🙂 🙂

I just finished updating my code and the param-build tests proved to be positive. So life is yet again full of flowers and sunshine. 😉

A mistake I’ ll never make again is define a static table of objects similar to defining an array of structures. Use parameterized constructors Vijay!!! /*Cursing myself right now!!! */

The mentors at Drizzle are so kind to help me out at each hurdle I encounter and I thank each and everyone of them. 🙂 Thanks guys. You are the BEST!!!

All I managed to do yesterday was upgrade my Ubuntu from 9.10 to 10.04 and frankly I consider it to be the best move I have made so far. 🙂 Lucid Lynx is awesome! Since I cleared up my previous Ubuntu I had to update my ssh keys at Launch Pad.

A point that Jay Pipes had forgotten to mention in his “Getting a Working C/C++ Development Environment for Developing Drizzle” is that the user has to sign into launchpad using the following command:

bzr launchpad-login username

Hope that he updates his post soon so that people new to drizzle development from here on can benefit from it. 🙂

So as of yesterday night, my drizzle development environment is all set and its “On your mark, Get set, CODE!!!”.


My Timeline

May 2010
M T W T F S S
« Apr   Jun »
 12
3456789
10111213141516
17181920212223
24252627282930
31  

I, Me and Myself

My Blog Stats

  • 6,462 hits

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 4 other followers