Scrapy is a nice python environment for web scraping, i.e. extracting information from web sites automatically by crawling them. It works best with anonymous data discovery, but nothing stops you from having active sessions as well. In fact, scrapy transparently manages cookies, which are usually used to track user sessions. Unfortunately, the sessions don't survive between runs. This, however, can be fixed quite easily by adding custom cookie middleware. Here … more »

After doing some web development work recently, I have clearly remembered why I hate JavaScript so much. Not only is it ugly as a language, lacking in type checking or even decent syntax for inheritance (actually faking it with various workarounds), but the actual fact of having effectively two separate code bases - front end and back end forces you to repeat quite a bit of code. And hunting for … more »

.NET's MemoryStream is a very convenient class. It allows you to use a byte[] array as storage, while accessing it via standard Stream API. Among other things, it allows you to work with a section of a larger byte array, which is very handy when different actions need to be taken for different slices. Unfortunately, when it comes to access the raw buffer stored in a MemoryStream instance, there is … more »

I use Ubuntu (a modern Linux distribution) as my main work station. Everything worked fine until I upgraded from version 14.10 to 15.04 of the OS. At this point strange things started to happen. After working for an hour or two my environment would freeze and stop responding altogether. Neither Ctrl-Alt-Del nor Ctrl-Alt-Backspace would produce any reaction. I couldn't connect to the machine remotely as well. Only hard reset would … more »

Not an uncommon task in multithreaded programming - what if you want to terminate a background thread, which is blocked at the moment, and then to make sure it has actually exited. In .NET, you can do the following: more »

What JVM-based web framework to choose? There seems to be so many out in the big world, and all claim to be better than the rest. Some examples of the frameworks of which I am personally aware: Lift Play Framework Grail Struts JSP and related technologies But there are many many more, with varying degrees of features, support, community sizes and popularity. Honestly, it is a daunting task to choose … more »

Earlier I wrote about issues with the new offer from Google - the "Inbox" mail service. Recently Google released updates for its Inbox web and Android clients. As a result, it works almost perfectly. In particular: Web interface works in Firefox as well as Chrome now. It looks much slicker than the old GMail, but I'm yet to be convinced it's more productive. This is quite different from Android, where … more »

values in Scala are, by specification, const, or final in Javaspeak. That means once assigned, they cannot be re-assigned. This doesn't mean that the assignment has to be something dull or simple. In fact, you can have a large piece of code doing all sorts of calculations with the result assigned to your value. Do it like this: more »

When it comes to SQL queries, there are two ways you can execute them from your program, whether it targets Android or not. One is ad hoc - create an SQL statement that finds your data (or manipulates it), with all the parameters of the actual query clauses already embedded in it. For example: more »

I was writing a piece of code in Scala, which was supposed to de-serialize some Java object, previously serialized elsewhere. However, I kept seeing exceptions like these: more »