Oleg Andreev

Process consists of a number of phases. Each phase provides a feedback on its performance.

Instead of defining some performance threshold for each phase to start optimization at and asking ourselves “when should we start optimizing this?”, we should rather ask ourselves “which phase is to be optimized now?”. That is, we should collect all feedback, sort it and start optimizing most important phases first. Naturally, we end the process when we are not getting any visible performance gains anymore.

This strategy can be applied to dynamic programming language runtime as well as to any other controllable process.

At each callsite (source code point, where method dispatch happens) we can count

1) number of calls to a method

2) time spent inside the method

3) time spent in callsite (total time across all methods called at the point).

Time can be measured either in byte code instructions, machine instructions or in microseconds.

Lets look at possible situations:

In real code, we don’t meet frequently called very slow methods: it is often done by bad design and could not be efficiently optimized in runtime. But this chart helps us to define a metric for “hot spot” identification: the place in the code, where we start runtime optimization.

Such “hotspot” metric would be callsite time * number of calls. The higher this number, the higher priority should be given to such callsite at optimization phase.

Why don’t we just start with a top-level method? If we start from the very top, we would spend enormous amount of time optimizing the whole program instead of really interesting parts.

Ruby symbols should be just immutable strings to avoid all that mess with string/symbol keys in option hashes. Just treat every literal string as immutable one. You could even keep this nice syntax with a colon to save some keystrokes.

So, basically:

String.should < ImmutableString
“symbol”.should == :symbol

Parser still fills a table of symbol when it encounters literal strings in the source code. So the unique number is just a string object_id.

When you compare two strings, you compare ids first and if they are not the same, you proceed with byte comparison. No need to convert :symbol.to_s.

How to make string a mutable one? Just #dup or #clone it.

“symbol”.dup.class.should == String

1. “Classic”: explicit require and manual dependencies resolution.

Pros: no bugs.
Cons: lazy programmers want to write and maintain less code.

2. “Rails method” (dependencies.rb): catch const_missing in runtime, guess the file name, load it.

Pros: zero-configuration, good automatic dependencies resolution.
Cons: file name guessing is a constraint, hard to debug runtime threading issues.

3. “Merb method” (load everything in a loop, reload failed files):

Pros: zero-configuration, moderate automatic dependencies resolution, no threading issues, issues are explicit and may arise in a bootstrap process only.
Cons: twice loaded constants yield warning, hidden repeated configuration bugs may appear.

4. “io language method” (load everything sorted by filename):

Pros: no bugs, zero-configuration, easy to understand, optional manual dependencies resolution.
Cons: 2-5% of files are named with ugly prefixes like A0_message.io (higher priority) or Z_exceptions.io (lower priority).

Note: no method saves you from manual resolution of the two-way referencing issue (when A references B and B references A). You still have to create a special “header” file to put bootstrap definitions into.

Conclusion: Merb method is essentially Io method with additional “feature” (which is buggy by nature) of double loading files in case of LoadError. If you name your files according to Io method and resolve all interdependencies manually, Merb will load them once and in the correct order.

We have a small team working with SVN. I like to write code under Git. Everyone is okay with command line except Sylvaine, our designer. She uses nice update/commit menu item in the Finder for her SVN-tracked PSD files.

So, here are all the facts:

0. We are working with Mac OS X.
1. I want all revisions stored on my machine.
2. I want good branch support for the source code.
3. I don’t need sophisticated branches for PSD-like documents.
4. Sylvaine probably does not want to open terminal to type “ga maquette.psd && gc ‘new block ’ && gush”. She wants to do this from the Finder with few clicks.
5. We have awful Mac GUIs for both Git and Mercurial (none for Git and halfass MacMercurial)
6. We have git-svn which works fine.

Result: keep individual documents under SVN, enjoy GUI and use git-svn in the Terminal. For source code use Git.

Everybody might be happy.

We already know CSV and TSV: comma-separated- and tab-separated- table formats. CSV looks ugly in human-readable files, TSV suffers from the invisible whitespaces and editor-dependent formatting.

I suggest another format for human-readable files: a list of records with fields separated with two or more spaces.

This helps to format the file with spaces only and avoid bloating file with quotes (and quote escape sequences). Text looks nice and light without unnecessary commas, quotes, backslashes and invisible tabs.

The only rule is:

1. Lines are separated by [\r\n]+
2. Fields are separated by \s\s+

The parser is obvious.

Actually, this format is already used in a variety of config files in *nix systems. Most noticeable example is /etc/fstab.

PS. Never use tab character where possible!

Some web2.0 guys might not understand quite well what each part of the “ehm-veh-ceh” pattern should and should not do. They should forget about “fat model and thin controller” misconcept. Today we try to help them. (Code should be light in either case.)

First of all, a simple example about “why it matters”:

A music player contains two volume controls and a sound stream. One control is a slider, another one is a “mute” button. Sound stream has an API with “volume”, “bass”, “track number” properties. If you try to modify sound stream directy within button and slider event callbacks, you will face a problem of UI consistency: when mute button is clicked, slider should be disabled or slide down. Ad hoc attempt is to bloat your callbacks with bunch of if-then-else branches checking all the possible cases in the whole UI in each callback!
A better way is to setup a single controller for all the volume controls with two actions: “triggerMute” and “setVolume(level)”. Then, subscribe each button to a volume property of the sound model, so that slider moves a handle according to the current volume level and mute button looks “unpressed” when the level is > 0. The final step is a controller which simply sets the volume level or triggers it. The only visible if-then-else branch here is the triggerMute method which should check the current volume level before changing it. However, this is an application-specific behavior which is out of scope of MVC pattern.

Generally speaking, your application is about “entities”. Given that:

1. Models are about state of entities. They describe consistent state and referential integrity within the whole system. Model must guarantee that any transition will result into consistent state. Model defines validations and constraints, but doesn’t define how transitions are performed (e.g. doesn’t check permissions). In case of database record, it should allow to change any values in any way, but ensures the consistency of the state just before transaction is committed. Perfect Model does not have “action” methods, but reader/writer accessors only.
When you write tests for a model (you write them, don’t you?) you should not describe an “operational process” with a model, but rather check that any possible update yields a valid state or appropriate errors are raised.

2. Controllers, on the other hand, are about actions. Controller does not bother about consistency of state (it may be stateless) but it specifies the control flow. That is, checks permissions, creates, deletes and moves entities around.
In a test for a controller you should describe initial state of the system, perform an action and then check the resulting state.

3. Views are about representation of the entities. View does not modify the entity, but it accesses its data, observes changes made to a model and generates events for a controller. When you test a view, you should define a set of entities (the input) and check their representation (the output).

The most simple and powerful MVC picture looks like that:

M -> V
V -> C
C -> M

View observes the state (M.) and updates itself when state is changed.
Controller observes the view and performs appropriate actions when some controls are triggered.

Bonus track. How to write perfect tests for a music player “volume feature”?

1. Model test verifies that volume level could not be below zero or over 9000.

2. Controller test checks several values for the “setVolume” method and two cases for the “triggerMute” method: with level = 0 and level = 1. Before the test, volume level is set to some value and after the action it is checked.

3. View test should check that the slider and the button generate events for the controller and observe events generated by the model.

Goddamn, developer MUST be independent. Switch on your brain, please. RTFM.

0. When ruby parser encounters assignment statement on some term looking like a local variable (a = 1, b ||= 2 etc.), it creates such variable. That’s why it knows later that “a” is a local variable, not a method call.

1. Sometimes this leads to a confusion:

def x;  :x; end

def y
  x = 42 if false # assignment is not executed
  x
end

y #=> nil (not :x as someone could predict)

# partial will have a local variable named "some_option"
partial(template, object, :some_option => 1)

# in this case there won't be a local variable "some_option"
partial(template, object)

$ скл Олег Андреев
Олег Андреев
Олега Андреева
Олегу Андрееву
Олега Андреева
Олегом Андреевым
Олеге Андрееве

$ gem sources -a http://gems.github.com
$ sudo gem install yaroslav-yandex_inflect

# ~/.bashrc
alias inf="ruby -rubygems -e'require \"yandex_inflect\"; puts YandexInflect.inflections(ARGV.join(\" \"))'"
alias скл=inf
alias склон=inf
alias склонять=inf
alias склонение=inf

$ sudo gem update --system
Password:
Updating RubyGems
Updating rubygems-update
Successfully installed rubygems-update-1.3.1
ERROR:  While executing gem ... (NameError)
    undefined local variable or method `remote_gemspecs' for
    #<Gem::Commands::UpdateCommand:0x11dfa8c>

There are always holy wars around Perl, Python, PHP, Ruby, Java and C#. We have enormous freedom on the server side comparing to a quirky browser environment. Why don’t we use Javascript on the server?

2-3 years ago server-side JS wouldn’t give us anything what wasn’t already available in other environments. But today we build rich applications with tons of code in javascript. Don’t we have some code to be shared with the server?

jQuery, prototype.js and MooTools may be used as a template engines, much flexible than <%%> and <??> tags, but also nicer and more lightweight than XSLT.

Selectors and transformations could be used both on the server side (to produce statically generated pages) and client side (for partial ajax updates).

Raw data interchange becomes easier. Also, you have common testing toolkit.

Recent JS implementations outperform Ruby, Python and Perl. Mostly, due to JS simplicity (no method_missing(), no classes and metaclasses etc.), but also because of enormous experience comparing with legacy server-side technologies. While Ruby, Python and Perl are used by X developers each, JS is used by (3*X + Y) developers plus hundreds of millions of mere mortals all around the world.

I believe, there’ll be web-server written for some open source JS engine very soon (Apache and Java don’t count).

Goto is not itself an absolute evil. Teaching youngsters to write a bad code is an absolute evil. Dirty tricks should be something what is clearly identified to be (sic!) “dirty”. Therefore they should be taught after good practices are successfully established.

1. Inline programmatic documentation with friendly command-line API
2. Ruby-like classes in 23 lines of code in Io

We have Bash already!

	case "$c$2" in
	--*=*) printf %s$'\n' "$c$2" ;;
	*.)    printf %s$'\n' "$c$2" ;;
	*)     printf %s$'\n' "$c$2 " ;;
	esac

Every programming language lacks some useful features, which other languages may provide. Many experienced programmers know several languages and try to port features and semantics from one language to another. It looks terribly wrong and inconvenient when one tries to write Java in Ruby, as well as vice versa. When you try to make a paper-like layout on the web for a newspaper you are shooting right into your foot and get nothing except buggy quirky page behavior and sleepless nights in fixing css bugs.
When you are trying to invent classes in JavaScript you end up with 10 different (incompatible) implementations with bizarre semantics (“where is the ‘this’?”).

The world is overcomplicated already. Meta programming (building all these frameworks, libraries and DSLs) is meant to extract and polish The Knowledge and minimize future efforts. Instead, we have to learn more and more things over time instead of keeping strong focus relying on our past achievements.

The first rule of thumb for a smart programmer: respect your language, choose appropriate tools for each task. Invent your own tools if you have to, but do not break already existing ones.

Thinking Forth helped me to realize the essential part of the Better Language. It is a scripting interface to libffi. That’s it.

The requirements for such interface are: portability, turing-completeness, single thread, simple grammar, no garbage collector (hence, no garbage).

Using this interface we could bootstrap a system environment for a language of our choice and easily connect important libraries like libcoroutine, llvm, libgarbagecollector and many others.

What is the shortest example code which shows great flaw in Ruby comparing to Java, for example? It is a literal “string”. That’s it. Every time interpreter hits the literal string, it creates a brand new object. The only useful thing is that it is already mutable, so you don’t have to copy it in rare cases when you want to modify it. But in reality you end up with such kind of code:

# Yeah, these constants look kind of tacky. Inside of tight loops,
# though, using them makes a small but measurable difference, and those
# small differences add up....
C_asterisk = '*'.freeze
C_empty = ''.freeze
C_header_close = 'HTTP/1.1 200 OK\r\nConnection: close\r\n'.freeze
C_header_keepalive = 'HTTP/1.1 200 OK\r\n'.freeze
C_slash = '/'.freeze
C_slashindex_html = '/index.html'.freeze
C1_0 = '1.0'.freeze
C1_1 = '1.1'.freeze
C_304 = "HTTP/1.1 304 Not Modified\r\n".freeze
Caos = 'application/octet-stream'.freeze
Cat = 'at'.freeze
Ccache_directory = 'cache_directory'.freeze
Ccache_extensions = 'cache_extensions'.freeze
...


The first thing “Better Language” should implement is source code being a first-class object with all the literals instantiated as immutable objects and referenced by this object. This helps to resolve a lot of issues with floats, strings and regexps.

However, the interesting things remain:
1. Complex literals should be recognized: like JSON (Arrays and Hashes) full of other literals should also be an immutable object attached to the source code object.
2. Literals which include variable references should turn into new objects creation on each method call, but only in a case when the variables are not already immutably binded to some values. This brings us to a whole world of functional programming from Lisp to Haskell.

Object-oriented design lets us avoid a bunch of “if-elsif-else” lines of code and use polymorphism instead. Sometimes it is extremely helpful to be able add or replace methods in core classes like strings, numbers and arrays. Ruby has such ability, but sometimes it turns out to be a nightmare. One library may redefine some String method for its own purpose and completely fuck up things in another library. In reality, library needs to redefine String class in that library’s namespace only. Let’s examine the possible ways to achive this:

1. In a pure OO-language only thing we have is message sends. When we write “string.size” we pass a message “size” to an object referenced by the local variable “string”.
Suppose, we have a library doing some low-level byte manipulations which relies on byte count behind “size” method. On the other hand, our application would like to deal with character count (one Cyrillic character is encoded with 2 bytes in UTF-8).
We want to keep the string in a same class (“String”) and also use the very same selector “size” for the different methods. So, how do we determine which method to choose, when class String receives the message “size”? The solution is to keep a thread-local stack of namespaces which is updated on each method call and ask it for a correct version of the “size” method. Maintaining such stack is not that hard, but method call becomes more complicated and slow: late binding overhead sums up with the namespace selection. Finally we have two virtual tables: for the eigenclass and for the namespaced extensions to that class.

2. In a statically typed language we know the type of the object from the lexical context. Thus, it is extremely easy to bind the correct version of the method according to that type using pattern-matching mechanisms, for example (see Haskell).

3. In case of polymorphic types, we have to curry type information with the objects and determine methods based on the lexical scope. However, we may reduce overhead if we use polymorphic virtual machine: such machine which injects itself into a compiled piece of code with its namespace in mind. In such case, message send is executed differently depending on the lexical scope of the code, where the send is performed. This allows us to avoid virtual table scans. But we have to waste memory keeping namespace-specific pieces of executable code.

I use Ruby everyday. I like C and Java for performance, but dislike their lack of expressiveness and build/configuration issues. I like Io’s garbage collector, coroutines and local state, but dislike its awful performance which cannot be optimized by design.
I generally hate Perl, but the idea of extensible grammar is interesting (remember E4X: why not to have an API to the parser to embed other grammars?).

In Ruby i’d like to extend core classes, but i don’t like to pollute other libraries with my helper methods. Thus, i need to narrow scope of some selectors to the lexical context.

I want pthreads, but with no implicit resource sharing: every thread must see only those resources, which are exposed for it explicitly. Communication between threads must be done through the efficient queues/pipes.

I want to compile arbitrary part of source code in runtime with arbitrary explicitly defined context. I want to have both portable bytecode and machine-dependent code and both serializable. Source code and runtime state must be serializable too.

At the core of the language, I want to have an extensible parser/interpreter, FFI and a compiler API.
The language should suit well for object-oriented programming, but should not be purely object-oriented.

I have removed EMRPC project from Rubyforge because it is not fun to work with their fucked up API. Github is much nicer (just update a gemspec and wait for 10 minutes). Add github to your gem sources list (if you haven’t done it already):

$ gem sources -a http://gems.github.com

$ sudo gem install oleganza-emrpc

I’m a software developer from St. Petersburg (Russia) currently living in Paris, France.

I work at Pierlis, private software company making websites and apps for Mac, iPhone and iPad.

Contacts

E-mail and jabber/gtalk: oleganza@gmail.com
Skype: olegandreev1986
Twitter: oleganza

Code

— github.com/oleganza
— pastie.org/search/?q=oleganza

936177597035857922347087286803058422491748052844532982535792