Every time I watch pravin talan shoots, I want to be a fashion photograher :)
Thursday, January 26, 2012
PHP Apache configurations
Server Request Response Flow-
Apache is web server, Its job is to receive the request and send the response. So if I opens a web page mydomain.com/test.html . request goes to server, server checks the mime type(.html in case) , executes the corresponding file and sends the response.
If I open a web page mydomain.com/test.php then also same flow happens. Request goes to server, server checks the mime type(.php), executes the test.php and send the response. But wait a sec, my server doest know how to execute this .php file? Here comes mod_php OR fastCGI. You might have heard about both of them.
Mod_Php - php sits inside the web server and gets loaded just like other modules of apache.
FastCGI- It is not part of web server. It is like a separate application which gets invoked by web server when it receives php mime type requests. Instead of creating a new process for each request, FastCGI uses persistent processes to handle a series of requests. These processes are owned by the FastCGI server, not the web server. FastCgi is a faster version of CGI. It is implemented by mod_fcgid and mod_fastcgi.
setup would be -
Apache MPM+ mod_fcgid
Apache MPM+ mod_fastcgi
How to set up fastcgi on apache -
https://wiki.archlinux.org/index.php/Apache_and_FastCGI
mod_fastcgi manage processes (not web server processes) through FastCGI Process Manager, fcgi-pm. This process manager is spawned by Apache at server initialization.
CGI Application Flow - CGI is a gateway to handle executable files (known as CGI scripts)
1. Apache got request GET /your_script
2. Your CGI app is loaded, in case of say Perl it's compiled first then loaded. To compile/translate it, the perl would have to be started. When it starts it loads dynamic libraries, which when loaded are pulling other dynamic libraries (in most cases).
3. When all is settled with Perl, your script is translated.
4. Now we're ready to execute our script.
5. Your script pulls the data out of environment, does it's job and calls exit.
6. The memory used by perl is freed, your compiled code is wasted. the libraries are unloaded and now we're waiting for step 1, just to make another similar job.
Quite clear that every time CGI app is loaded, lot of depending libraries are loaded to serve one request and on completion of request CGI app is unloaded. Process repeats for any next request. Repeating loading and unloading is time consuming.
FAST CGI Flow -
1. Apache got request GET /your_script
2. Your CGI app is loaded, in case of say Perl it's compiled first then loaded. To compile/translate it, the perl would have to be started. When it starts it loads dynamic libraries, which when loaded, are pulling other dynamic libraries (in most cases).
3. When all is settled with Perl, your script is translated.
4. Now we're ready to execute our script.
5. Your script pulls the data out of environment, does it's job and it goes Back to step 4.
Difference Mod_php AND FAST CGI -
Conceptually it is quite clear that Mod_php is faster as it is bundled with apache and preloaded so web server is able to serve the request in less time. But this comes with High memory usage as these libraries are always pre loaded even when apache is serving static html files. If you dont have memory problem (you do have it on a shared server) then Mod_php is good to go. But as Mod_php is part of apache php config settings changes wont be reflected without restarting apache (unless you do a dynamic mod_php installation).
PHP-FPM (FastCGI Process Manager) - is an alternative PHP FastCGI implementation with some additional features useful for sites of any size, especially busier sites.
Apache worker + mod_fastcgi + php-fpm
when we use mod_fcid or mod_fastcgi each process has its's own OPCode cache but in PHP-FPM that opcode cache is shared across processes. Php-fpm supports socket connection (for local server when web server and cgi server are on same machine) as well as TCP/IP connection (for remote servers). socket connection is definitely faster.
know more at http://php-fpm.org/about/
APACHE MPM Worker and APACHE MPM PREFORK-
Ever wonder how apache handles requests. It has to start a process to serve a request. To manage simultaneous requests it keeps a pool of processes running and keeps forking/killing them to maintain maximum process limit. lets understand this in detail-
APACHE MPM Worker - It implements a hybrid multi process multi threaded server. It uses threads to serve requests and able to serve more requests with less resource as compared to process based server (Prefork).
- based on apache 2.0
- lower memory consumption and higher performance
How Worker MPM works-
MPM Prefork - It is a non thread server and appropriate for sites that need to avoid threading for compatibility with non-thread-safe libraries.(You might have come across options to download thread-safe/non thread-safe versions of libraries several times.) It isolates each request and handles in separate process so that a problem with a single request will not affect any other.
- based on apache 1.3
- stable and secure
- high memory consumption and low performance
How Prefork works
A single control process is responsible for launching child processes which listen for connections and serve them when they arrive. Apache always tries to maintain several spare or idle server processes, which stand ready to serve incoming requests. In this way, clients do not need to wait for a new child processes to be forked before their requests can be served.
The StartServers, MinSpareServers, MaxSpareServers, and MaxClients regulate how the parent process creates children to serve requests. In general, Apache is very self-regulating, so most sites do not need to adjust these directives from their default values. Sites which need to serve more than 256 simultaneous requests may need to increase MaxClients, while sites with limited memory may need to decrease MaxClients to keep the server from thrashing.
Magento PerformanceTuning -
Magento is highly resource extensive. The web server settings play a big role in fine tuning, reducing the load time for website and increasing the number of requests handled.
Mod_php OR FastCGI - well simple answer - if you dont have memory and resource problems then go for mod_php, it is no doubt faster but eats up lot of resources.
Worker OR Prefork - we are using php so we might have to deal with non thread safe third party libraries. PHP5 is thread-safe, but PHP extensions aren't all thread-safe. And Magento is a PHP application like any other, chances are that you are using some PHP extensions somewhere. So it's considered harmfull to run a PHP application on a worker-mpm. so using Prefork is a better option.
+ Nishant
Apache is web server, Its job is to receive the request and send the response. So if I opens a web page mydomain.com/test.html . request goes to server, server checks the mime type(.html in case) , executes the corresponding file and sends the response.
If I open a web page mydomain.com/test.php then also same flow happens. Request goes to server, server checks the mime type(.php), executes the test.php and send the response. But wait a sec, my server doest know how to execute this .php file? Here comes mod_php OR fastCGI. You might have heard about both of them.
Mod_Php - php sits inside the web server and gets loaded just like other modules of apache.
FastCGI- It is not part of web server. It is like a separate application which gets invoked by web server when it receives php mime type requests. Instead of creating a new process for each request, FastCGI uses persistent processes to handle a series of requests. These processes are owned by the FastCGI server, not the web server. FastCgi is a faster version of CGI. It is implemented by mod_fcgid and mod_fastcgi.
setup would be -
Apache MPM+ mod_fcgid
Apache MPM+ mod_fastcgi
How to set up fastcgi on apache -
https://wiki.archlinux.org/index.php/Apache_and_FastCGI
mod_fastcgi manage processes (not web server processes) through FastCGI Process Manager, fcgi-pm. This process manager is spawned by Apache at server initialization.
CGI Application Flow - CGI is a gateway to handle executable files (known as CGI scripts)
1. Apache got request GET /your_script
2. Your CGI app is loaded, in case of say Perl it's compiled first then loaded. To compile/translate it, the perl would have to be started. When it starts it loads dynamic libraries, which when loaded are pulling other dynamic libraries (in most cases).
3. When all is settled with Perl, your script is translated.
4. Now we're ready to execute our script.
5. Your script pulls the data out of environment, does it's job and calls exit.
6. The memory used by perl is freed, your compiled code is wasted. the libraries are unloaded and now we're waiting for step 1, just to make another similar job.
Quite clear that every time CGI app is loaded, lot of depending libraries are loaded to serve one request and on completion of request CGI app is unloaded. Process repeats for any next request. Repeating loading and unloading is time consuming.
FAST CGI Flow -
1. Apache got request GET /your_script
2. Your CGI app is loaded, in case of say Perl it's compiled first then loaded. To compile/translate it, the perl would have to be started. When it starts it loads dynamic libraries, which when loaded, are pulling other dynamic libraries (in most cases).
3. When all is settled with Perl, your script is translated.
4. Now we're ready to execute our script.
5. Your script pulls the data out of environment, does it's job and it goes Back to step 4.
Difference Mod_php AND FAST CGI -
Conceptually it is quite clear that Mod_php is faster as it is bundled with apache and preloaded so web server is able to serve the request in less time. But this comes with High memory usage as these libraries are always pre loaded even when apache is serving static html files. If you dont have memory problem (you do have it on a shared server) then Mod_php is good to go. But as Mod_php is part of apache php config settings changes wont be reflected without restarting apache (unless you do a dynamic mod_php installation).
PHP-FPM (FastCGI Process Manager) - is an alternative PHP FastCGI implementation with some additional features useful for sites of any size, especially busier sites.
Apache worker + mod_fastcgi + php-fpm
when we use mod_fcid or mod_fastcgi each process has its's own OPCode cache but in PHP-FPM that opcode cache is shared across processes. Php-fpm supports socket connection (for local server when web server and cgi server are on same machine) as well as TCP/IP connection (for remote servers). socket connection is definitely faster.
know more at http://php-fpm.org/about/
APACHE MPM Worker and APACHE MPM PREFORK-
Ever wonder how apache handles requests. It has to start a process to serve a request. To manage simultaneous requests it keeps a pool of processes running and keeps forking/killing them to maintain maximum process limit. lets understand this in detail-
APACHE MPM Worker - It implements a hybrid multi process multi threaded server. It uses threads to serve requests and able to serve more requests with less resource as compared to process based server (Prefork).
- based on apache 2.0
- lower memory consumption and higher performance
How Worker MPM works-
A single control process (the parent) is responsible for launching child processes. Each child process creates a fixed number of server threads as specified in the ThreadsPerChild directive, as well as a listener thread which listens for connections and passes them to a server thread for processing when they arrive.
Apache always tries to maintain a pool of spare or idle server threads, which stand ready to serve incoming requests. In this way, clients do not need to wait for a new threads or processes to be created before their requests can be served. The number of processes that will initially launched is set by the StartServers directive. Then during operation, Apache assesses the total number of idle threads in all processes, and forks or kills processes to keep this number within the boundaries specified by MinSpareThreads and MaxSpareThreads. Since this process is very self-regulating, it is rarely necessary to modify these directives from their default values. The maximum number of clients that may be served simultaneously (i.e., the maximum total number of threads in all processes) is determined by the MaxClients directive. The maximum number of active child processes is determined by the MaxClients directive divided by the ThreadsPerChild directive.
Apache always tries to maintain a pool of spare or idle server threads, which stand ready to serve incoming requests. In this way, clients do not need to wait for a new threads or processes to be created before their requests can be served. The number of processes that will initially launched is set by the StartServers directive. Then during operation, Apache assesses the total number of idle threads in all processes, and forks or kills processes to keep this number within the boundaries specified by MinSpareThreads and MaxSpareThreads. Since this process is very self-regulating, it is rarely necessary to modify these directives from their default values. The maximum number of clients that may be served simultaneously (i.e., the maximum total number of threads in all processes) is determined by the MaxClients directive. The maximum number of active child processes is determined by the MaxClients directive divided by the ThreadsPerChild directive.
MPM Prefork - It is a non thread server and appropriate for sites that need to avoid threading for compatibility with non-thread-safe libraries.(You might have come across options to download thread-safe/non thread-safe versions of libraries several times.) It isolates each request and handles in separate process so that a problem with a single request will not affect any other.
- based on apache 1.3
- stable and secure
- high memory consumption and low performance
How Prefork works
A single control process is responsible for launching child processes which listen for connections and serve them when they arrive. Apache always tries to maintain several spare or idle server processes, which stand ready to serve incoming requests. In this way, clients do not need to wait for a new child processes to be forked before their requests can be served.
The StartServers, MinSpareServers, MaxSpareServers, and MaxClients regulate how the parent process creates children to serve requests. In general, Apache is very self-regulating, so most sites do not need to adjust these directives from their default values. Sites which need to serve more than 256 simultaneous requests may need to increase MaxClients, while sites with limited memory may need to decrease MaxClients to keep the server from thrashing.
Magento PerformanceTuning -
Magento is highly resource extensive. The web server settings play a big role in fine tuning, reducing the load time for website and increasing the number of requests handled.
Mod_php OR FastCGI - well simple answer - if you dont have memory and resource problems then go for mod_php, it is no doubt faster but eats up lot of resources.
Worker OR Prefork - we are using php so we might have to deal with non thread safe third party libraries. PHP5 is thread-safe, but PHP extensions aren't all thread-safe. And Magento is a PHP application like any other, chances are that you are using some PHP extensions somewhere. So it's considered harmfull to run a PHP application on a worker-mpm. so using Prefork is a better option.
+ Nishant
Wednesday, January 25, 2012
Password hashing And encryption
I happened to write this while I was diging how Magento stores password.
well magento stores hashed password separated by a two character salt in db. A sample looks like -
Now lets understand some cryptography Hashing and encryption.
Password hashing and encryption are two different things. Hashing is one way function while encryption is two way. Reverse encryption (called decryption) is possible but you can not reverse hash and generate your original string back. You can find another string which has same hash value by brute force methods.
Collision - Hashing generates a fixed length output (called as message digest, checksum) for all input values which makes it possible to have two different strings (S1#S2) but H(S1)=H(S2). That is called collision. Probability of collision depends upon the algorithm and length of hashed output.
To secure passwords, cryptographic hash algorithms are used(MD5, SHA-1, SHA-2).
But Plain hashing is easily defeated using a dictionary attack, where an attacker just pre-hashes every word in a dictionary (or every combination of characters up to a certain length), then uses this new dictionary to look up hashed passwords. Salting, Key stretching, HMAC can be used to strengthen your hashing, will discuss that later.
How hash algorithms work - Most algorithms(MD5, SHA-1, SHA-2, SHA-3) are based on Block Ciphers. Input string is divided into fixed length blocks and then padded, encrypted with keys, processed with complex operations (ADD/AND/XOR/ROTATE) sequentially to generate final output. That makes it hard to break. For example computing 40*11 is easy but factorizing 440 and generating 40 and 11 is a bit hard because of multiple possibilities. That is what it makes hard to break.
Salting -
hash = md5(password + salt);Instead of directly hashing the input you can add a pinch of salt to make it spicy :)
salt could be any digit random string, which you store along with the hashed value. You can store your passwords in db as hash:salt. For authentications when user enters a password, it's hash is calculated using entered password and salt(from db) and validated against the hash(from db).
Key Stretching - You add the salt and hash it for n times. You have to store that count(n) as well, along with salt.
Don't use
in the loop, it will only increase the collision probability.
Common Hashing Algorithms-
CRC32 (Cyclic redundancy check) - Very simple hash function.Produces 32 bit long hash value and typically expressed as 8 digit hexadecimal string.
MD-5 - MD5 produces 128 bit(16 byte) hash value. It is typically expressed as 32 digit hexadecimal string(each hexadecimal digit takes 4 bits). This algorithm has been broken and found vulnerable.
SHA-1 - SHA-1 produces 160 bit hash value. It is typically expressed as 40 digit hexadecimal string. It's approximately two-three times slower then MD5 algorithm This algorithm is also broken and found vulnerable.
SHA-2(SHA-512) - SHA-2 produces 512 bit hash value. typically expressed as 128 digit hexadecimal string. This is not yet broken.
HMAC - Unlike the other hashes mentioned above, HMAC (Hashed Message Authentication Code) is a key dependant hash. HMACs are useful when authentication but not secrecy of a message is required.Current HMAC specification is defined as
The cryptographic strength of the HMAC depends upon the size of the secret key that is used. The most common attack against HMACs is brute force to uncover the secret key. HMACs are substantially less affected by collisions than their underlying hashing algorithms alone. Therefore, HMAC-MD5 does not suffer from the same weaknesses that have been found in MD5.
How does HMAC work?
How to break a Hash -
If you want to find a given plaintext for a certain hash there are two simple methods:
- Hash each plaintext one by one, until you find the hash.
- Hash each plaintext one by one, but store each generated hash in a sorted table so that you can easily look the hash up later without generating the hashes again.
Simple.. huh? well lets discuss that.
What is Rainbow Table-
Hash Function converts plain text into hashed output which cannot be dehashed. But how abut if there is a mapping table which stores every possible mapping from Hash Value to Plain Text, then you just need to see if your hash value exists in this table or not and you will get the plain text for that hash. That table is known as Rainbow table. Obviouslly there is no such pre existing table so this is how you use Rainbow table to break a hash -
Inputs-
hash password(to be broken)
input password is 6 digit (numeric)
hash alogorithm used- Md5
Output-
input password
lets say my password is 493823
This input is feed to MD5 and output is feed to Reduction function, and the cycle goes on. It represents a chain from starting input text to ending hash(you can choose to end at any hash). The table only stores the starting plaintext, and the final hash you choose to end with, and so a chain "containing" millions of hashes can be represented with only a single starting plaintext, and a single finishing hash.
Now we can use these chains to break the given hash password and find the input password.
The algorithm is:
+Nishant
References- http://kestas.kuliukas.com/RainbowTables/
well magento stores hashed password separated by a two character salt in db. A sample looks like -
353dc2ba6108461cf3468184bdd0e174:QP
split = 353dc2ba6108461cf3468184bdd0e174:QP
split = 353dc2ba6108461cf3468184bdd0e174:QP
magentoPass = split[0];
salt = split[1];
salt = split[1];
##Authenticate user entered manager.
# is used for concatanation
if( md(5) [split[1]#userenteredpasswd])==split[0])
{
# User Authenticated
}
Now lets understand some cryptography Hashing and encryption.
Password hashing and encryption are two different things. Hashing is one way function while encryption is two way. Reverse encryption (called decryption) is possible but you can not reverse hash and generate your original string back. You can find another string which has same hash value by brute force methods.
Collision - Hashing generates a fixed length output (called as message digest, checksum) for all input values which makes it possible to have two different strings (S1#S2) but H(S1)=H(S2). That is called collision. Probability of collision depends upon the algorithm and length of hashed output.
 |
| Collision |
To secure passwords, cryptographic hash algorithms are used(MD5, SHA-1, SHA-2).
But Plain hashing is easily defeated using a dictionary attack, where an attacker just pre-hashes every word in a dictionary (or every combination of characters up to a certain length), then uses this new dictionary to look up hashed passwords. Salting, Key stretching, HMAC can be used to strengthen your hashing, will discuss that later.
How hash algorithms work - Most algorithms(MD5, SHA-1, SHA-2, SHA-3) are based on Block Ciphers. Input string is divided into fixed length blocks and then padded, encrypted with keys, processed with complex operations (ADD/AND/XOR/ROTATE) sequentially to generate final output. That makes it hard to break. For example computing 40*11 is easy but factorizing 440 and generating 40 and 11 is a bit hard because of multiple possibilities. That is what it makes hard to break.
Salting -
hash = md5(password + salt);
salt could be any digit random string, which you store along with the hashed value. You can store your passwords in db as hash:salt. For authentications when user enters a password, it's hash is calculated using entered password and salt(from db) and validated against the hash(from db).
Key Stretching - You add the salt and hash it for n times. You have to store that count(n) as well, along with salt.
hash = md5(password + salt);
for (i = 0; i < 1000; i++)
{
hash = md5(hash + password + salt);
}
Don't use
hash = md5(hash)
Common Hashing Algorithms-
CRC32 (Cyclic redundancy check) - Very simple hash function.Produces 32 bit long hash value and typically expressed as 8 digit hexadecimal string.
MD-5 - MD5 produces 128 bit(16 byte) hash value. It is typically expressed as 32 digit hexadecimal string(each hexadecimal digit takes 4 bits). This algorithm has been broken and found vulnerable.
SHA-1 - SHA-1 produces 160 bit hash value. It is typically expressed as 40 digit hexadecimal string. It's approximately two-three times slower then MD5 algorithm This algorithm is also broken and found vulnerable.
SHA-2(SHA-512) - SHA-2 produces 512 bit hash value. typically expressed as 128 digit hexadecimal string. This is not yet broken.
HMAC - Unlike the other hashes mentioned above, HMAC (Hashed Message Authentication Code) is a key dependant hash. HMACs are useful when authentication but not secrecy of a message is required.Current HMAC specification is defined as
# used for concatenation
H(key1 ∥ H(key2 ∥ message)).
The cryptographic strength of the HMAC depends upon the size of the secret key that is used. The most common attack against HMACs is brute force to uncover the secret key. HMACs are substantially less affected by collisions than their underlying hashing algorithms alone. Therefore, HMAC-MD5 does not suffer from the same weaknesses that have been found in MD5.
How does HMAC work?
Lets Define-
ipad = inner padding: the byte 0x36 repeated the same number of times as the block size
opad = outer padding: the byte 0x5c repeated the same number of times as the block size
text = the message we wish to compute the HMAC over.
The length of the key should be less than or equal to the block size (64 bytes for MD5 and SHA-1), though greater than the size of the message digest (16 bytes for MD5, 20 bytes for SHA-1). If the key length is greater than the block size, a (fixed length) hash of the key should be used.
To compute HMAC over the data 'text' the following steps are performed:
1. the key is appended with zero bytes until it equals the block size in length.
2. the key is XORed with ipad
3.text is appended to the result of 2
4.the hash algorithm is applied to the result of 3
5.the key is XORed with opad
6.the result of 4 is appended to the result of 5
7.the hash algorithm is applied to the result of 6.How to break a Hash -
If you want to find a given plaintext for a certain hash there are two simple methods:
- Hash each plaintext one by one, until you find the hash.
- Hash each plaintext one by one, but store each generated hash in a sorted table so that you can easily look the hash up later without generating the hashes again.
Simple.. huh? well lets discuss that.
What is Rainbow Table-
Hash Function converts plain text into hashed output which cannot be dehashed. But how abut if there is a mapping table which stores every possible mapping from Hash Value to Plain Text, then you just need to see if your hash value exists in this table or not and you will get the plain text for that hash. That table is known as Rainbow table. Obviouslly there is no such pre existing table so this is how you use Rainbow table to break a hash -
Inputs-
hash password(to be broken)
input password is 6 digit (numeric)
hash alogorithm used- Md5
Output-
input password
lets say my password is 493823
MD5("493823") -> "222f00dc4b7f9131c89cff641d1a8c50"
I have a Reduction function which takes first 6 numbers from the last generated hash. R("222f00dc4b7f9131c89cff641d1a8c50") -> "222004".
This input is feed to MD5 and output is feed to Reduction function, and the cycle goes on. It represents a chain from starting input text to ending hash(you can choose to end at any hash). The table only stores the starting plaintext, and the final hash you choose to end with, and so a chain "containing" millions of hashes can be represented with only a single starting plaintext, and a single finishing hash.
Now we can use these chains to break the given hash password and find the input password.
The algorithm is:
- start-Look for the hash in the list of final hashes, if it is there break out of the loop.
- If it isn't there reduce the hash into another plaintext, and hash the new plaintext.
- Goto the start point.
- If the hash matches one of the final hashes, the chain for which the hash matches the final hash contains the original hash.
+Nishant
References- http://kestas.kuliukas.com/RainbowTables/
Saturday, January 21, 2012
One night at Indian Wedding
 |
| Ref- theperfectpose.com |
It was 20th Jan 2012. Seeing weather report in your cozy heated room in a chilled winter and actually experiencing that in out are totally different. Minimum temperature touched to 5.9, lowest of this season and I happened to be at the one of my very best friend's sister's wedding.
I was greeted by two lovely ladies at the entrance, escorted to coffee stall and a smart gentleman in Tuxedos asked me- Hot Coffee or Cold Sir?, taking cold coffee in such a cold was an interesting but not a good idea which I realized next day. Then mesmerizing smell of fried Mushrooms took my attention. Again a sweet girl accompanied by a gentleman with lovely mushrooms in his hand, delightfully asked me and I couldn't resist. I happened to notice a secluded GolGappee stall, a mustache man, white gloves in his hand, sitting on a chair handed me the plat. Golgappe were nicely covered to protect them from any dust. Eating golgappe in such a sophisticated way took all the fun away, It was nothing compared to a roadside spicy golgappe shop.
Even Management and Wedding Planning business is sure a recession free and forever booming business. One thought was banging my head all night why people spend so much money over a one day event, when the family of bride and groom are too busy and tired to enjoy the most important day of their's and their kid's life.
While I was busy thinking over it, barat came and every one was busy greeting them. The boys were shivering in jackets and the girls were not giving a damn about the cold. The universal truth is after all true that ladies don't fell the cold or I would rather say they are hot. The bride and groom are now on stage and all are staring at them, I dont know how does it feel to be sitting at that chair and seeing all people staring at you, laughing, dancing, making comments, thinking twice before drinking and eating any thing to avoid natural calls, smiling at every one unnecessarily, thinking if you are looking good enough to have good photos. Isn't it too much ? I am sure he must have thought of running away from all this at least once. Now it was time of family photographs. Guest already eaten enough(they had to ,to compensate for the gift they had given), were happy to see bride and grooms finally at stage. They formed a civilized queue to give their best wishes to married couple, say good bye and head back to home. This family photo session followed by couple's hilarious photo shoot with carefully chosen poses by photographers, finally ended around 2 AM in the morning.
Time to have dinner. All waiters were lined up to give the bride and groom the best meal of their best day. Nothing was left to make them feel like a young prince and princess, everyone following them and making sure to take care of every small thing. Nicely chopped salad and fruits, juices and mock-tails, tons of vegetables half never heard of, different kinds of breads, Chinese, Italian, Thai food. I was overwhelmed. I couldn't help but noticing a nice gentleman ordering someone on his walky to start making Jalebis, and those nicely curved crispy Jalebis with Hot milk on that chilled day was in front of me few minutes later. It was time for sweets. I didnt even bother about other sweet dishes till I happened to encounter the chocolate cake the same which got over as told to me at the cake stall earlier :) . It was sure a day of that sweet couple.
Dinner was soon over and I found myself into a small , beautifully decorated mandap, covered with lot of gas heaters to keep the place warm, I was tired enough to wander, think or speak, was sitting quietly at a heated corner sipping coffee one after other, listening to vows Groom and Bride made to each other. I happened to hear them at lot of weddings but never happened to remember them , dont know it was the funny way Panditji was describing them or I was too tired to divert my focus any where else.
I realized it was not about putting up a big event to show off to society but to make this society witness to the vows this newly wed couple was taking before tieing their wedding knot.
It was soon dawn, Bride was given a tearful farewell. I came to home thinking about I am not far away from the day anyhow will try to keep my new year resolution of being bachelor this year.
Software Project Lifecycle
I never realised criticality involved with Software Project Lifecycle until I became part of -
-Undefined Requirements
- Rework and Rework
- Never Ending Projects
- Bugs
- Late Deliveries
- Bad Cash flow
Above mentioned points are pain points of any Software Development Startup. At every step of project lifecycle there is good chance of miscommunication/gaps which could ultimate effect the deliveries and company economy or I better say a startup economy. There has to be a full proof process which can take care of all transitions smoothly. Before devising that process I am sure that the process has to serve following -
1. The Requirement Specs should be as clear as possible.
2. Considering requirements will vary there has to be a team which is in touch with client and knows the project right from start.
3. Task distribution is easy, but their status also need to be monitored.
4. Feedbacks with client are important at every milestone.
5. Quality Check before delivery
6. Most Important - Delivery on time.
Keeping above points in minds I believe there have to be following teams -
Team A - First Communication point. Client gives a brief requirement and asks for a quote.
A does few meetings with clients and prepares a requirement doc[R-Doc]. Now A should feed this input to another team-B and gets the effort estimate on top of which A can generate the cost estimate and negotiate with client.
Team B goes through the requirement doc, discusses with Team A, do some magical analysis and outputs an effort and feature doc [F-Doc] and send back to Team A. This team also takes care of Specs.
Now this F-Doc comes to Team C which takes care of implementing it.
Once things are done Team B does a Quality Check (Team B knows the project best).
There is Team D which takes care of timings and make sure deliveries are done on time by poking Team C to do some work actually :).
Well that seems simple. Now lets make this process more clear by discussing these teams.
Team A is Business Team, People in this team should have good sales skills to even sell Used Newspapers at millions and Business analytic skills to better understand the client Business requirement and prepare our R-Doc( There can be multiple revisions.) Now problem is our Team A is not smart enough to tell the effort estimate. Well they always think that tough job is done so things should be done by FRIDAY :)
Team B is Product development Team. It understands that magical R-Doc and generates F-Doc :) and also tells how much effort will it take. This team should sure have good technology skills to estimate the work correctly, good analytic skills to quality check the implemented work.
Team C is Engineering/Technology team. It is a Black Box , It converts F-Doc into a nice software application. It definetely should have good development, coding skills and debugging skills. After developing this team is supposed to test the application and submit to Team D . Team D asks for Team B for quality check and approval and give a go ahead for final deployment..
Team D- is Delivery Team, it takes care of time and makes sure things get done on time. It also takes care of demos and feedbacks.
P Team(Project Teams) - They are sub teams of our Engineering Team.There must be multiple projects going around. So there should be a dedicated team for every project. This dedicated team will have Techno-Manager/Project Manager + Project Lead + Developer(s). Team B should involve P teams at-least the techno-manager and tech lead while preparing F-Doc.
Feedbacks/Requirement Change - Change in requirements are certain to come. Team D takes care of it.The Client Feedback and Change in Requirements are sent to Team B for approval and modification in F-Doc, things come back to Engineering team. Project Manger and Project Lead decides in which release the newly requested features will go in and they are rolled out accordingly.
The above process is explained below.
This process has made my life simple :)
-Undefined Requirements- Rework and Rework
- Never Ending Projects
- Bugs
- Late Deliveries
- Bad Cash flow
Above mentioned points are pain points of any Software Development Startup. At every step of project lifecycle there is good chance of miscommunication/gaps which could ultimate effect the deliveries and company economy or I better say a startup economy. There has to be a full proof process which can take care of all transitions smoothly. Before devising that process I am sure that the process has to serve following -
1. The Requirement Specs should be as clear as possible.
2. Considering requirements will vary there has to be a team which is in touch with client and knows the project right from start.
3. Task distribution is easy, but their status also need to be monitored.
4. Feedbacks with client are important at every milestone.
5. Quality Check before delivery
6. Most Important - Delivery on time.
Keeping above points in minds I believe there have to be following teams -
Team A - First Communication point. Client gives a brief requirement and asks for a quote.
A does few meetings with clients and prepares a requirement doc[R-Doc]. Now A should feed this input to another team-B and gets the effort estimate on top of which A can generate the cost estimate and negotiate with client.
Team B goes through the requirement doc, discusses with Team A, do some magical analysis and outputs an effort and feature doc [F-Doc] and send back to Team A. This team also takes care of Specs.
Now this F-Doc comes to Team C which takes care of implementing it.
Once things are done Team B does a Quality Check (Team B knows the project best).
There is Team D which takes care of timings and make sure deliveries are done on time by poking Team C to do some work actually :).
Well that seems simple. Now lets make this process more clear by discussing these teams.
Team A is Business Team, People in this team should have good sales skills to even sell Used Newspapers at millions and Business analytic skills to better understand the client Business requirement and prepare our R-Doc( There can be multiple revisions.) Now problem is our Team A is not smart enough to tell the effort estimate. Well they always think that tough job is done so things should be done by FRIDAY :)
Team B is Product development Team. It understands that magical R-Doc and generates F-Doc :) and also tells how much effort will it take. This team should sure have good technology skills to estimate the work correctly, good analytic skills to quality check the implemented work.
Team C is Engineering/Technology team. It is a Black Box , It converts F-Doc into a nice software application. It definetely should have good development, coding skills and debugging skills. After developing this team is supposed to test the application and submit to Team D . Team D asks for Team B for quality check and approval and give a go ahead for final deployment..
Team D- is Delivery Team, it takes care of time and makes sure things get done on time. It also takes care of demos and feedbacks.
P Team(Project Teams) - They are sub teams of our Engineering Team.There must be multiple projects going around. So there should be a dedicated team for every project. This dedicated team will have Techno-Manager/Project Manager + Project Lead + Developer(s). Team B should involve P teams at-least the techno-manager and tech lead while preparing F-Doc.
Feedbacks/Requirement Change - Change in requirements are certain to come. Team D takes care of it.The Client Feedback and Change in Requirements are sent to Team B for approval and modification in F-Doc, things come back to Engineering team. Project Manger and Project Lead decides in which release the newly requested features will go in and they are rolled out accordingly.
The above process is explained below.
This process has made my life simple :)
Thursday, January 19, 2012
Add Items in Order Email template in Magento
Magento Admin panel provides a way to add email template in System->TransactionalEmails.
There are few existing templates like -
New Order Guest Email
New Order User Email
Shipment Email
Invoice,CreditMemo Email
Registration Email etc
app/design/fronend/default/default/layout/sales.xml has following section -
<!--
Email layouts section
-->
<sales_email_order_items>
I added a new template information inside above block -
<block type="sales/order_email_items" name="newtemplate" template="email/order/newtemplate.phtml"></block>
Now lets open app/design/frontend/default/default/template/email/order/items.phtml.
Just add -
<?php $i=0; foreach ($_order->getAllItems() as $_item): ?>
<?php endforeach; ?>
Now lets create newtemplate.phtml at app/design/frontend/default/default/template/email/order/newtemplate.phtml
this newtemplate basically gets items from another table for which I have created a Model, that model has function getAllItems(), put following code in your newtemplate.phtml -
<?php
$order = $this->getOrder();
$itemUrl = $this->getUrl();
$itemModel= Mage::getModel('mymodule/mymodel');
$extraItems = $itemModel->getAllItems($order);
$i = 0;
?>
<?php if ($extraItems ): ?>
<?php foreach ($extraItems as $extraItem):?>
<tbody<?php echo $i%2 ? ' bgcolor="#F6F6F6"' : '' ?>>
<tr>
<td align="left" valign="top" style="font-size:11px; padding:3px 9px; border-bottom:1px dotted #CCCCCC;">
<?php echo '<a href="'.$itemUrl.'">' ?>
<?php echo $this->htmlEscape($extraItem->getproduct_name());?>
</a>
</td>
<td align="left" valign="top" style="font-size:11px; padding:3px 9px; border-bottom:1px dotted #CCCCCC;">
<?php echo $this->htmlEscape($extraItem->getsku());?> </td>
<td align="center" valign="top" style="font-size:11px; padding:3px 9px; border-bottom:1px dotted #CCCCCC;"> <?php echo $extraItem->getqty_ordered() * 1;?> </td>
<td align="right" valign="top" style="font-size:11px; padding:3px 9px; border-bottom:1px dotted #CCCCCC;"> 0 </td>
</tr>
</tbody>
<?php $i++; ?>
<?php endforeach; ?>
<?php endif; ?>
That is it.
Email Order Item template will include these extra items :)
There are few existing templates like -
New Order Guest Email
New Order User Email
Shipment Email
Invoice,CreditMemo Email
Registration Email etc
{{layout handle="sales_email_order_items" order=$order}}
above line in Order email template gets item information.app/design/fronend/default/default/layout/sales.xml has following section -
<!--
Email layouts section
-->
<sales_email_order_items>
<block type="sales/order_email_items" name="items" template="email/order/items.phtml">
------------------------
------------------------
</block>
</sales_email_order_items>
I added a new template information inside above block -
<block type="sales/order_email_items" name="newtemplate" template="email/order/newtemplate.phtml"></block>
Now lets open app/design/frontend/default/default/template/email/order/items.phtml.
Just add -
<?php echo $this->getChildHtml('newtemplate') ?>
after following foreach loop.<?php $i=0; foreach ($_order->getAllItems() as $_item): ?>
<?php endforeach; ?>
Now lets create newtemplate.phtml at app/design/frontend/default/default/template/email/order/newtemplate.phtml
this newtemplate basically gets items from another table for which I have created a Model, that model has function getAllItems(), put following code in your newtemplate.phtml -
<?php
$order = $this->getOrder();
$itemUrl = $this->getUrl();
$itemModel= Mage::getModel('mymodule/mymodel');
$extraItems = $itemModel->getAllItems($order);
$i = 0;
?>
<?php if ($extraItems ): ?>
<?php foreach ($extraItems as $extraItem):?>
<tbody<?php echo $i%2 ? ' bgcolor="#F6F6F6"' : '' ?>>
<tr>
<td align="left" valign="top" style="font-size:11px; padding:3px 9px; border-bottom:1px dotted #CCCCCC;">
<?php echo '<a href="'.$itemUrl.'">' ?>
<?php echo $this->htmlEscape($extraItem->getproduct_name());?>
</a>
</td>
<td align="left" valign="top" style="font-size:11px; padding:3px 9px; border-bottom:1px dotted #CCCCCC;">
<?php echo $this->htmlEscape($extraItem->getsku());?> </td>
<td align="center" valign="top" style="font-size:11px; padding:3px 9px; border-bottom:1px dotted #CCCCCC;"> <?php echo $extraItem->getqty_ordered() * 1;?> </td>
<td align="right" valign="top" style="font-size:11px; padding:3px 9px; border-bottom:1px dotted #CCCCCC;"> 0 </td>
</tr>
</tbody>
<?php $i++; ?>
<?php endforeach; ?>
<?php endif; ?>
That is it.
Email Order Item template will include these extra items :)
Wednesday, January 18, 2012
Add sorting attribute in product list page in Magento
You can see the top pager in product list page in which there is sort by dropdown.
Magento by default provides following sort by attributes for any category -
1. Position
2. Name
3. Price
4. Brand
Lets say I want to add an extra attribute net discount for sorting. We have to create an attribute in magento admin panel (Catalog->Manage Attributes) , lets name it discount and set sorting in product listing in frontend properties as yes as shown in following image.
Now go to Catalog->Manage Categories in Magento Admin Panel. Click on Any Category->Display Settings tab and you can see your new attribute under Available Product Listing sorting by -
Now when you have added this attribute it would be available on all product list pages but what would happen when one clicks on it??
Do we have to write action, controller, model for that , luckily we have better option. We just need to modify catalog product collection (app/code/core/Mage/Catalog/Model/Resource/Eav/Mysql4/Product/Collection.php) You can create a local copy in (app/code/local/Mage/Catalog/Model/Resource/Eav/Mysql4/Product/Collection.php)
Inside function addAttributeToSort we have to add an following code -
if ($attribute == 'discount' && $storeId != 0) {
$this->getSelect()->joinLeft(
array('price_index_new' => $this->getTable('catalog_product_index_price')),
"price_index_new.entity_id = e.entity_id AND price_index_new.website_id =1 AND price_index_new.customer_group_id =0",
array("((price_index_new.price-price_index_new.final_price)/(price_index_new.price)) as netdiscount")
);
$this->getSelect()->order("netdiscount {$dir}");
return $this;
}
That is easy :)
Now I think when I click sort by discount I should by default get the results in desc order rather than ascending order, what should be done for that?
We just need to modify a template file. (app/design/fronend/base/default/template/catalog/product/list/toolbar.phtml) create a copy - (app/design/fronend/default/default/template/catalog/product/list/toolbar.phtml)
around line 8- you can find
<?php foreach($this->getAvailableOrders() as $_key=>$_order): ?>
<option value="<?php if ($_key=='discount') echo $this->getOrderUrl($_key, 'desc'); else echo $this->getOrderUrl($_key, 'asc'); ?>"<?php if($this->isOrderCurrent($_key)): ?> selected="selected"<?php endif; ?>>
<?php echo $this->__($_order) ?>
</option>
<?php endforeach; ?>
That is it. Happy Sorting :)
Magento by default provides following sort by attributes for any category -
1. Position
2. Name
3. Price
4. Brand
Lets say I want to add an extra attribute net discount for sorting. We have to create an attribute in magento admin panel (Catalog->Manage Attributes) , lets name it discount and set sorting in product listing in frontend properties as yes as shown in following image.
Now go to Catalog->Manage Categories in Magento Admin Panel. Click on Any Category->Display Settings tab and you can see your new attribute under Available Product Listing sorting by -
Now when you have added this attribute it would be available on all product list pages but what would happen when one clicks on it??
Do we have to write action, controller, model for that , luckily we have better option. We just need to modify catalog product collection (app/code/core/Mage/Catalog/Model/Resource/Eav/Mysql4/Product/Collection.php) You can create a local copy in (app/code/local/Mage/Catalog/Model/Resource/Eav/Mysql4/Product/Collection.php)
Inside function addAttributeToSort we have to add an following code -
if ($attribute == 'discount' && $storeId != 0) {
$this->getSelect()->joinLeft(
array('price_index_new' => $this->getTable('catalog_product_index_price')),
"price_index_new.entity_id = e.entity_id AND price_index_new.website_id =1 AND price_index_new.customer_group_id =0",
array("((price_index_new.price-price_index_new.final_price)/(price_index_new.price)) as netdiscount")
);
$this->getSelect()->order("netdiscount {$dir}");
return $this;
}
That is easy :)
Now I think when I click sort by discount I should by default get the results in desc order rather than ascending order, what should be done for that?
We just need to modify a template file. (app/design/fronend/base/default/template/catalog/product/list/toolbar.phtml) create a copy - (app/design/fronend/default/default/template/catalog/product/list/toolbar.phtml)
around line 8- you can find
<?php foreach($this->getAvailableOrders() as $_key=>$_order): ?>
add if/else here as following - <?php foreach($this->getAvailableOrders() as $_key=>$_order): ?>
<option value="<?php if ($_key=='discount') echo $this->getOrderUrl($_key, 'desc'); else echo $this->getOrderUrl($_key, 'asc'); ?>"<?php if($this->isOrderCurrent($_key)): ?> selected="selected"<?php endif; ?>>
<?php echo $this->__($_order) ?>
</option>
<?php endforeach; ?>
That is it. Happy Sorting :)
Subscribe to:
Posts (Atom)