Posts for the month of May 2007

Vi Module Meets ErlyBird

There have been several Erlang development tools: Erlang module for Emacs, for vim, and Erlide for Eclipse. Why I write another Erlang IDE based on NetBeans?

Erlang for Emacs runs smoothly on my computer, but the distel module can not communicate with Erlang node on my Windows XP, that means I can not have the auto-completion, go to declaration features working; Erlang for vim is not a complete IDE yet; Erlide hangs on my Windows XP too. So I write ErlyBird.

But I'm actually a vi fun, so I just download and install the excellent jVi module to ErlyBird, which is a fully functional vi environment with good performance. There is an article talking about vi integration with NetBeans, which can also be applied to ErlyBird.

I'm now with fun with Vi on ErlyBird on my daily job.

The biggest issue for ErlyBird currently is the rendering performance, which causes performance slowing down if you run ErlyBird a while. I'm not sure if this issue depends on Generic Language Framework module of NetBeans. After I get the new laptop which with 2G memory next week, I may do some profile analysis.

I've also written some code to talk with Erlang Node from ErlyBird, everything looks smooth too.

I'll fly to San Francisco next week, to meet my new and old friends there.

It seems that this has been a world you should mix Vi/Netbeans, Java/Erlang, Beijing/Vancouver/San Francisco, whatever, together? A dynamical, colorful, multi-culture world, you have to look for the truths carefully, continually.

ErlyBird 0.11.2 released - An Erlang IDE based on NetBeans

I'm pleased to announce ErlyBird 0.11.2, an Erlang IDE based on NetBeans.

This is a bug-fix, stabilization release. Since I tightly modified GSF/GLF modules of NetBeans, this release will only provide all-in-one IDE package, which is in size of 14.8M.

To download, please go to:

To install:

  1. Unzip to somewhere. For Windows user, executee 'bin/erlybird.exe'. For *nix user, 'bin/erlybird'.
  2. Check/set your OTP path. From [Tools]->[Options], click on 'Miscellanous', then expand 'Erlang Installation', fill in the full path of your 'erl.exe' or 'erl' file. For instance: "C:/erl/bin/erl.exe"
  3. The default -Xmx option for jvm is set to 256M, if you want to increase, open the config file of ErlyBird that is located at etc/erlybird.conf, set -J-Xmx in line of 'default_options'

The status of ErlyBird is still Alpha, feedback and bug reports are welcome.


  • Indexing will skip too big files according to the max memeory. This avoids ErlyBird to hang when indexing.
  • If erl/erl.exe is under the environment path, ErlyBird will try to set Erlang Installation path automatically.
  • Including function args in completion suggestion.
  • Various bugs fixes especially for stabilization.

From Rails to Erlyweb - Part III

3. The Magic Behind Erlyweb

With dynamic typing, hot code swapping, built-in parsing and compilation tools, Erlang is also suitable for dynamic meta-programming. Erlyweb uses a small convenient tool smerl to generate db record CRUD code automatically.

The music example on shows the magic:

Define a simple musician.erl module (just one line here):


Then, you will get a long list functions for musician module after erlyweb:compile("apps/music"). Sounds similar to Rails.

I like to watch magic show, but I can't stand that I do not know the things behind magic in programming. For Rails, the magic behind it always makes me headache, it's too difficult to follow who, where, some code are injected into a class or module. But in Erlang, it's super easy.

I add a simple function to erlyweb_app.erl (please see my previous post):


decompile(AppName, Beam) when is_atom(AppName) ->
    decompile(atom_to_list(AppName), Beam);
decompile(AppName, Beam) when is_list(AppName) ->
    BinFilename = "./apps/" ++ AppName ++ "/ebin/" ++ atom_to_list(Beam),
    io:format("Beam file: ~s~n", [BinFilename]),
    {ok, {_, [{abstract_code, {_, AC}}]}} = beam_lib:chunks(BinFilename, [abstract_code]),
    SrcFilename = "./apps/" ++ AppName ++ "_" ++ atom_to_list(Beam),
    {ok, S} = file:open(SrcFilename ++ ".erl", write),
    io:fwrite(S, "~s~n", [erl_prettypr:format(erl_syntax:form_list(AC))]). 

Now type erlyweb_decompile(music, musician) under the Erlang shell, I get a file: music_musician.erl under folder myproject/apps/ (I put these decompiled source files under /myproject/apps/ to avoid that they are auto-compiled to beams by erlyweb again ):

-file("musician", 1).


-export([relations/0, fields/0, table/0, type_field/0,
	 db_table/0, db_field/1, is_new/1, is_new/2,
	 get_module/1, to_iolist/1, to_iolist/2,
	 field_to_iolist/2, new/0, new_with/1, new_with/2,
	 new_from_strings/1, set_fields/2, set_fields/3,
	 set_fields_from_strs/2, field_from_string/2, save/1,
	 insert/1, update/1, update/2, delete/1, delete_id/1,
	 delete_where/1, delete_all/0, transaction/1,
	 before_save/1, after_save/1, before_delete/1,
	 after_delete/1, after_fetch/1, find/2, find_first/2,
	 find_max/3, find_range/4, find_id/1, aggregate/4,
	 count/0, get/2, set_related_one_to_many/2,
	 find_related_one_to_many/2, find_related_many_to_one/4,
	 find_related_many_first/4, find_related_many_max/5,
	 find_related_many_range/6, aggregate_related_many/6,
	 do_save/1, do_delete/1, field_names_for_query/0,
	 field_names_for_query/1, field_to_iolist/1, set/3,
	 db_pk_fields/0, get_pk_fk_fields/0, get_pk_fk_fields2/0,
	 db_fields/0, db_field_names/0, db_field_names_str/0,
	 db_field_names_bin/0, db_num_fields/0, id/1, id/2,
	 name/1, name/2, birth_date/1, birth_date/2,
	 instrument/1, instrument/2, bio/1, bio/2, new/4,
	 driver/0, count/3, count/1, count/2, count_with/2,
	 avg/3, avg/1, avg/2, avg_with/2, min/3, min/1, min/2,
	 min_with/2, max/3, max/1, max/2, max_with/2, sum/3,
	 sum/1, sum/2, sum_with/2, stddev/3, stddev/1, stddev/2,
	 stddev_with/2, find/0, find/1, find_with/1,
	 find_first/0, find_first/1, find_first_with/1,
	 find_max/1, find_max/2, find_max_with/2, find_range/2,
	 find_range/3, find_range_with/3]).

relations() -> erlydb_base:relations().

fields() -> erlydb_base:fields().

table() -> erlydb_base:table().

type_field() -> erlydb_base:type_field().

db_table() -> erlydb_base:db_table(musician).

db_field(FieldName) ->
    erlydb_base:db_field(musician, FieldName).

is_new(Rec) -> erlydb_base:is_new(Rec).

is_new(Rec, Val) -> erlydb_base:is_new(Rec, Val).

get_module(Rec) -> erlydb_base:get_module(Rec).

to_iolist(Recs) ->
    erlydb_base:to_iolist(musician, Recs).

to_iolist(Recs, ToIolistFun) ->
    erlydb_base:to_iolist(musician, Recs, ToIolistFun).

field_to_iolist(Val, _Field) ->
    erlydb_base:field_to_iolist(Val, _Field).

new() -> erlydb_base:new(musician).

new_with(Fields) ->
    erlydb_base:new_with(musician, Fields).

new_with(Fields, ToFieldFun) ->
    erlydb_base:new_with(musician, Fields, ToFieldFun).

new_from_strings(Fields) ->
    erlydb_base:new_from_strings(musician, Fields).

set_fields(Record, Fields) ->
    erlydb_base:set_fields(musician, Record, Fields).

set_fields(Record, Fields, ToFieldFun) ->
    erlydb_base:set_fields(musician, Record, Fields,

set_fields_from_strs(Record, Fields) ->
    erlydb_base:set_fields_from_strs(musician, Record,

field_from_string(ErlyDbField, undefined) ->
    erlydb_base:field_from_string(ErlyDbField, undefined).

save(Rec) -> erlydb_base:save(Rec).

insert(Recs) -> erlydb_base:insert(Recs).

update(Props) -> erlydb_base:update(musician, Props).

update(Props, Where) ->
    erlydb_base:update(musician, Props, Where).

delete(Rec) -> erlydb_base:delete(Rec).

delete_id(Id) -> erlydb_base:delete_id(musician, Id).

delete_where(Where) ->
    erlydb_base:delete_where(musician, Where).

delete_all() -> erlydb_base:delete_all(musician).

transaction(Fun) ->
    erlydb_base:transaction(musician, Fun).

before_save(Rec) -> erlydb_base:before_save(Rec).

after_save(Rec) -> erlydb_base:after_save(Rec).

before_delete(Rec) -> erlydb_base:before_delete(Rec).

after_delete({_Rec, Num}) ->
    erlydb_base:after_delete({_Rec, Num}).

after_fetch(Rec) -> erlydb_base:after_fetch(Rec).

find(Where, Extras) ->
    erlydb_base:find(musician, Where, Extras).

find_first(Where, Extras) ->
    erlydb_base:find_first(musician, Where, Extras).

find_max(Max, Where, Extras) ->
    erlydb_base:find_max(musician, Max, Where, Extras).

find_range(First, Max, Where, Extras) ->
    erlydb_base:find_range(musician, First, Max, Where,

find_id(Id) -> erlydb_base:find_id(musician, Id).

aggregate(AggFunc, Field, Where, Extras) ->
    erlydb_base:aggregate(musician, AggFunc, Field, Where,

count() -> erlydb_base:count(musician).

get(Idx, Rec) -> erlydb_base:get(Idx, Rec).

set_related_one_to_many(Rec, Other) ->
    erlydb_base:set_related_one_to_many(Rec, Other).

find_related_one_to_many(OtherModule, Rec) ->
    erlydb_base:find_related_one_to_many(OtherModule, Rec).

find_related_many_to_one(OtherModule, Rec, Where,
			 Extras) ->
    erlydb_base:find_related_many_to_one(OtherModule, Rec,
					 Where, Extras).

aggregate_related_many_to_one(OtherModule, AggFunc, Rec,
			      Field, Where, Extras) ->
					      AggFunc, Rec, Field, Where,

add_related_many_to_many(JoinTable, Rec, OtherRec) ->
    erlydb_base:add_related_many_to_many(JoinTable, Rec,

remove_related_many_to_many(JoinTable, Rec, OtherRec) ->
    erlydb_base:remove_related_many_to_many(JoinTable, Rec,

remove_related_many_to_many_all(JoinTable, OtherTable,
				Rec, Where, Extras) ->
						OtherTable, Rec, Where, Extras).

find_related_many_to_many(OtherModule, JoinTable, Rec,
			  Where, Extras) ->
					  JoinTable, Rec, Where, Extras).

aggregate_related_many_to_many(OtherModule, JoinTable,
			       AggFunc, Rec, Field, Where, Extras) ->
					       JoinTable, AggFunc, Rec, Field,
					       Where, Extras).

find_related_many_first(Func, Rec, Where, Extras) ->
    erlydb_base:find_related_many_first(Func, Rec, Where,

find_related_many_max(Func, Rec, Num, Where, Extras) ->
    erlydb_base:find_related_many_max(Func, Rec, Num, Where,

find_related_many_range(Func, Rec, First, Last, Where,
			Extras) ->
    erlydb_base:find_related_many_range(Func, Rec, First,
					Last, Where, Extras).

aggregate_related_many(Func, AggFunc, Rec, Field, Where,
		       Extras) ->
    erlydb_base:aggregate_related_many(Func, AggFunc, Rec,
				       Field, Where, Extras).

do_save(Rec) -> erlydb_base:do_save(Rec).

do_delete(Rec) -> erlydb_base:do_delete(Rec).

field_names_for_query() ->

field_names_for_query(UseStar) ->
    erlydb_base:field_names_for_query(musician, UseStar).

field_to_iolist(Val) ->

set(Idx, Rec, Val) -> setelement(Idx, Rec, Val).

db_pk_fields() ->
    erlydb_base:db_pk_fields([{erlydb_field, id, "id",
			       <<105, 100>>, int, 11, integer, text_field,
			       false, primary, undefined, identity}]).

get_pk_fk_fields() ->
    erlydb_base:get_pk_fk_fields([{id, musician_id}]).

get_pk_fk_fields2() ->
    erlydb_base:get_pk_fk_fields2([{id, musician_id1,

db_fields() ->
    erlydb_base:db_fields([{erlydb_field, id, "id",
			    <<105, 100>>, int, 11, integer, text_field, false,
			    primary, undefined, identity},
			   {erlydb_field, name, "name", <<110, 97, 109, 101>>,
			    varchar, 20, binary, text_field, true, undefined,
			    undefined, undefined},
			   {erlydb_field, birth_date, "birth_date",
			    <<98, 105, 114, 116, 104, 95, 100, 97, 116, 101>>,
			    date, undefined, date, text_field, true, undefined,
			    undefined, undefined},
			   {erlydb_field, instrument, "instrument",
			    <<105, 110, 115, 116, 114, 117, 109, 101, 110,
			    [<<103, 117, 105, 116, 97, 114>>,
			     <<112, 105, 97, 110, 111>>,
			     <<100, 114, 117, 109, 115>>,
			     <<118, 111, 99, 97, 108, 115>>],
			    binary, select, true, undefined, undefined,
			   {erlydb_field, bio, "bio", <<98, 105, 111>>, text,
			    undefined, binary, text_area, true, undefined,
			    undefined, undefined}]).

db_field_names() ->
    erlydb_base:db_field_names([id, name, birth_date,
				instrument, bio]).

db_field_names_str() ->
    erlydb_base:db_field_names_str(["id", "name",
				    "birth_date", "instrument", "bio"]).

db_field_names_bin() ->
    erlydb_base:db_field_names_bin([<<105, 100>>,
				    <<110, 97, 109, 101>>,
				    <<98, 105, 114, 116, 104, 95, 100, 97, 116,
				    <<105, 110, 115, 116, 114, 117, 109, 101,
				      110, 116>>,
				    <<98, 105, 111>>]).

db_num_fields() -> erlydb_base:db_num_fields(5).

id(Rec) -> erlydb_base:get(3, Rec).

id(Rec, Val) -> setelement(3, Rec, Val).

name(Rec) -> erlydb_base:get(4, Rec).

name(Rec, Val) -> setelement(4, Rec, Val).

birth_date(Rec) -> erlydb_base:get(5, Rec).

birth_date(Rec, Val) -> setelement(5, Rec, Val).

instrument(Rec) -> erlydb_base:get(6, Rec).

instrument(Rec, Val) -> setelement(6, Rec, Val).

bio(Rec) -> erlydb_base:get(7, Rec).

bio(Rec, Val) -> setelement(7, Rec, Val).

new(name, birth_date, instrument, bio) ->
    {musician, true, undefined, name, birth_date,
     instrument, bio}.

driver() ->
			[{last_compile_time, {{1980, 1, 1}, {0, 0, 0}}},
			 {outdir, "apps/music/ebin"}, debug_info, report_errors,
			 report_warnings, {erlydb_driver, mysql}]}).

count(Field, Where, Extras) ->
    erlydb_base:aggregate(musician, count, Field, Where,

count(Field) ->
    erlydb_base:aggregate(musician, count, Field, undefined,

count(Field, Where) ->
    erlydb_base:aggregate(musician, count, Field, Where,

count_with(Field, Extras) ->
    erlydb_base:aggregate(musician, count, Field, undefined,

avg(Field, Where, Extras) ->
    erlydb_base:aggregate(musician, avg, Field, Where,

avg(Field) ->
    erlydb_base:aggregate(musician, avg, Field, undefined,

avg(Field, Where) ->
    erlydb_base:aggregate(musician, avg, Field, Where,

avg_with(Field, Extras) ->
    erlydb_base:aggregate(musician, avg, Field, undefined,

min(Field, Where, Extras) ->
    erlydb_base:aggregate(musician, min, Field, Where,

min(Field) ->
    erlydb_base:aggregate(musician, min, Field, undefined,

min(Field, Where) ->
    erlydb_base:aggregate(musician, min, Field, Where,

min_with(Field, Extras) ->
    erlydb_base:aggregate(musician, min, Field, undefined,

max(Field, Where, Extras) ->
    erlydb_base:aggregate(musician, max, Field, Where,

max(Field) ->
    erlydb_base:aggregate(musician, max, Field, undefined,

max(Field, Where) ->
    erlydb_base:aggregate(musician, max, Field, Where,

max_with(Field, Extras) ->
    erlydb_base:aggregate(musician, max, Field, undefined,

sum(Field, Where, Extras) ->
    erlydb_base:aggregate(musician, sum, Field, Where,

sum(Field) ->
    erlydb_base:aggregate(musician, sum, Field, undefined,

sum(Field, Where) ->
    erlydb_base:aggregate(musician, sum, Field, Where,

sum_with(Field, Extras) ->
    erlydb_base:aggregate(musician, sum, Field, undefined,

stddev(Field, Where, Extras) ->
    erlydb_base:aggregate(musician, stddev, Field, Where,

stddev(Field) ->
    erlydb_base:aggregate(musician, stddev, Field,
			  undefined, undefined).

stddev(Field, Where) ->
    erlydb_base:aggregate(musician, stddev, Field, Where,

stddev_with(Field, Extras) ->
    erlydb_base:aggregate(musician, stddev, Field,
			  undefined, Extras).

find() ->
    erlydb_base:find(musician, undefined, undefined).

find(Where) ->
    erlydb_base:find(musician, Where, undefined).

find_with(Extras) ->
    erlydb_base:find(musician, undefined, Extras).

find_first() ->
    erlydb_base:find_first(musician, undefined, undefined).

find_first(Where) ->
    erlydb_base:find_first(musician, Where, undefined).

find_first_with(Extras) ->
    erlydb_base:find_first(musician, undefined, Extras).

find_max(Max) ->
    erlydb_base:find_max(musician, Max, undefined,

find_max(Max, Where) ->
    erlydb_base:find_max(musician, Max, Where, undefined).

find_max_with(Max, Extras) ->
    erlydb_base:find_max(musician, Max, undefined, Extras).

find_range(First, Max) ->
    erlydb_base:find_range(musician, First, Max, undefined,

find_range(First, Max, Where) ->
    erlydb_base:find_range(musician, First, Max, Where,

find_range_with(First, Max, Extras) ->
    erlydb_base:find_range(musician, First, Max, undefined,

With this decompiled file, you get all things clearly behind the magic, such as, you have pair getter/setter functions for each field, for example:

Musician = musician:find({name, 'like' "Caoyuan Mus"}),
%% get the 'name' field of record Musician
Name = musician:name(Musician), 
%% set the 'name' field to "new name" and bind to a new record Musician1.
Musician1 = musician:name(Musician, "new name"),
%% Or,
Musician2 = musician:set_fields(Musician, {name, "new name"}, {birth_day, "1940/10/9"}),
%% then save one of them

Finally, some notices for new comer to Erlang and Erlyweb:

  1. In Erlang, the Variable can only be bound(set value) once , so, only Musician1 and Musician2 have the "new name", Musician will keep the original value
  2. For efficiency reason, if the field is varchar/text type, the getter will return a binary rather than string, which can be printed on browser directly in Erlyweb, but, if you want to use it as a string, you can apply binary_to_list(Name) on it.

From Rails to Erlyweb - Part II

Updated Aug 23: Please see From Rails to Erlyweb - Part II Manage Project - Reloaded

Updated July 15: store the database configuration in <opaque> session of yaws.conf

Updated May 2: erlweb:compile(AppDir::string(), Options::[option()]) has option: {auto_compile, Val}, where Val is 'true', or 'false'. In case of development, you can turn on {auto_compile, true}. So, you only need to run erlyweb_app:boot(myapp) once.

2. Manage project

Erlyweb provides erlyweb:compile(App, ..) to compile the source files under app directory. To start an app, you usually should erlydb:start(mysql, ....) and compile app files first. To make life easy, you can put some scripting like code under myproject\script directory. Here's my project source tree:

  + apps
  |   + myapp
  |       + ebin   
  |       + include
  |       + nbproject
  |       + src
  |           + components
  |           + lib
  |           + services
  |       + test
  |       + www
  + config
  |   * yaws.conf
  + script
      + ebin
      + src
           * erlyweb_app.erl

Where, config/yaws.conf contains the confsiguration that will copy/paste to your real yaws.conf file. Here's mine:

## This is the configuration of apps that will copy/paste to your yaws.conf.

ebin_dir = D:/myapp/trunk/script/ebin
ebin_dir = D:/myapp/trunk/apps/myapp/ebin

<server localhost>
	port = 8000
	listen =
	docroot = D:/myapp/trunk/apps/myapp/www
	appmods = </myapp, erlyweb>
		appname = myapp
		hostname = "localhost"
		username = "mememe"
		password = "pwpwpw"
		database = "myapp_development"	

You may have noticed, all beams under D:/myapp/trunk/script/ebin and D:/myapp/trunk/apps/myapp/ebin will be auto-loaded by yaws.

erlyweb_app.erl is the boot scripting code, which will be used to start db connection and compile the code. Currently I run these scripts manually. I'll talk later.





%% @doc call back funtion when yaws start an app  
%% @see man yaws.conf
%%      start_mod = Module
%%          Defines  a  user  provided  callback  module.  At startup of the
%%          server, Module:start/1 will  be  called.   The  #sconf{}  record
%%          (defined  in  yaws.hrl) will be used as the input argument. This
%%          makes it possible for  a  user  application  to  syncronize  the
%%          startup  with  the  yaws  server as well as getting hold of user
%%          specific  configuration  data,  see  the  explanation  for   the
%%           context.
start(ServerConf) ->
    Opaque = ServerConf#sconf.opaque,
    AppName  = proplists:get_value("appname",  Opaque),
    Database = proplists:get_value("database", Opaque),
    DBConf = [{database, Database},
              {hostname, proplists:get_value("hostname", Opaque)},
              {username, proplists:get_value("username", Opaque)},
              {password, proplists:get_value("password", Opaque)}],
    io:fwrite(user, "Starting app ~s using database:~n~s~n", [AppName, Database]),

start_db(DBConf) ->
    erlydb:start(mysql, DBConf).

main([AppName]) ->
main(_) ->

boot(AppName) ->
    build(AppName, true).

build(AppName) -> build(AppName, false).
build(AppName, AutoCompile) when is_atom(AppName) -> build(atom_to_list(AppName), AutoCompile);
build(AppName, AutoCompile) when is_list(AppName) ->
    compile(AppName, [debug_info, {auto_compile, AutoCompile}]).

compile(AppName, Options) ->
    % Retrieve source header paths from yaws server configuration.  We don't know
    % how to get yaws.hrl here yet, so we manually write matching rule here.
    {ok, GC, _} = yaws_server:getconf(),
    {gconf, _, _, _, _, _, _, _, _, _, _, _, _, _, Incl, _, _, _, _, _} =  GC,
    %?Debug("paths: ~p", [Incl]),
        "./apps/" ++ AppName,
        [{erlydb_driver, mysql}, {i, Incl}] ++ Options).

decompile(AppName, Beam) when is_atom(AppName) -> decompile(atom_to_list(AppName), Beam);
decompile(AppName, Beam) when is_list(AppName) ->
    BinFilename = "./apps/" ++ AppName ++ "/ebin/" ++ atom_to_list(Beam),
    io:format("Beam file: ~s~n", [BinFilename]),
    {ok, {_, [{abstract_code, {_, AC}}]}} = beam_lib:chunks(BinFilename, [abstract_code]),
    SrcFilename = "./apps/" ++ AppName ++ "_" ++ atom_to_list(Beam),
    %% do not with ".erl" ext? otherwise will be compiled by ealyweb
    {ok, S} = file:open(SrcFilename ++ ".erl", write), 
    io:fwrite(S, "~s~n", [erl_prettypr:format(erl_syntax:form_list(AC))]).

usage() ->
    io:format("usage: erlyweb_app AppName\n"),

To build it,

> erlc -I /opt/local/lib/yaws/include erlyweb_app.erl

The erlyweb_app.erl is almost escript ready, but I use it as module functions currently. It's pre-compiled and erlyweb_app.beam is placed under script/ebin

So, I start myapp by steps:

cd \myproject
yaws -i -sname yaws
1> erlyweb_app:build(myapp).

The erlyweb_app.erl is almost escript ready, but I use it as module functions currently. It's pre-compiled and erlyweb_app.beam is placed under script/ebin

After I made changes to myapp, I run above erlyweb_app:build(myapp). again, then everything is up to date.

This is surely not the best way to handle the write-compile-run-test cycle, I'll improve the scripting to let starting yaws as a node, then hot-swap the compiled code to it.

It's a good experience to play with Rails, I like rake db:migrate, script, config folders of Rails. And Grails also brings some good idea to manage web app project tree. I'll try to bring them into project manager of ErlyBird.

Next part, I'll talk about the magic behind Erlyweb, and why I prefer the magic of Erlyweb to Rails.