It is funny, but when a programmer is developing a product, he rarely thinks about the question: Could 2000 people press simultaneously one button at the same time? It turns out they can. Curiously enough the most engines were written by such programmers that do not work well under heavy loads. Who would have thought that just one additional INSERT or unwritten index or a recursive function curve could raise significantly the load averages.
This article will describe how the developers of the project managed to squeeze out of a single server with the Pentium 4 HT / 512Mb RAM the maximum performance, holding simultaneously 700 users on the forum and 120000 on tracker. Also, it will give some details about Highload.
Let us learn how to scale your application without having any
Experience, it is very difficult. Now there are many websites that are devoted to these issues, unfortunately, there is no solution that is suitable for all cases. We still need to find solutions ourselves, which are suitable for our requirements. Just as I do.
Several years ago, my boss came to me and said: “We have a new project for you, namely to transfer a website, which already has 1 million visitors per a month. You need to move this website and make sure that traffic could grow in the future without any problems”. I was already an experienced programmer, but I did not have any experience in the field of scalability. I had to learn scalability in the hard way.
The documentation for a non-blocking web server Tornado beautifully described as it copes with a great load, and generally it is the crown of mankind creation in the field of non-blocking servers. This is partly true. But when building the complex applications beyond the scope of “one more chat” reveals many non-obvious and subtle points about which we should know before run into the pitfalls. The club developers of intellectual games “Trellis” willing to share their thoughts about the pitfalls.
We are talking about a second branch of python, the latest version of Tornado 1.2.1, and postgresql, which is connected over psycopg2.
Many programmers love to use a pattern of singleton for easy access to the application class. If we think about the future horizontal scaling that is not recommended to do it. The request object will give you a thread-safe application that can be used safely.
2147483647 (231-1), Mersenne prime is the maximum possible value for 32-bit integer that is the largest integer that can be written in 32 bits.
What does this have to do with the phone numbers? Ironically, it has the most direct relation. It turns out that a significant number of American programmers are developing the systems in order to optimize it, where numbers are stored on the server in the form of 32-bit integers. Thus, the maximum possible number is equal to (214) 748-3647 in the United States, where 214 is a code of Dallas. When we enter into the database greater value then it is stored the maximum possible number of 2147483647.
If we conduct a search on the Internet, we can find hundreds of the phone books from all over America, which refer to the same number in Dallas. We can only sympathize to the owner of this phone number.
How could not the project’s clients to notice the mistake on the part of developers? Probably many of them did the business in the region, where the code is less than 214, so that the other phone numbers just did not get into the database. Maybe the developers convinced someone that this is the best way to optimize: in such form the numbers take up less space than when are stored in the form of individual characters. In fact many are obsessed with the optimization. The lessons of Y2K did not go well for all; moreover, a new generation of programmers grew up who do not remember Y2K at all.