Skip to content

Help Denis

… to keep his hair.

Today I stumbled upon the “Made With Meteor” web page. It’s basically a simple web app that allows you to promote your own Meteor app. At the same time you are helping the ecosystem, because you can showcase what is possible with Meteor. The top-voted app is Pintask, a really cool Trello clone. Clone is not the right word here, since it’s outperforming the original in many aspects.

After registering and playing around with the app, I found an email by one of it’s creators Denis. He contacted me and asked for help. Since he made a (silly) bet with his colleague that the app will have 200+ votes on April the 30th (on “Made with Meteor”). If not Denis will shave his head off. So please help him, to keep his hair. All you need is a Meteor developer account and than please click on the tiny arrow.

Promises

Well. After a quite long break. I’m back. At least sort of. I don’t promise anything. And nothing you might have to say, or not, will change this. How could the latter …

It’s hard to tell, why I actually stopped blogging.

The great and beautiful blogger Rarasaur once asked why so many blogs simply disappear or are neglected after some inspired and flourishing first month. I did just that and still, I don’t have a simple answer for her.

I liked blogging – putting this words together feels awkward, it’s really been a while since I wrote something – and I still like it.

My enthusiasm started to fall apart as I was forced into a quite long blogging break, aka summer holidays. Afterwards I couldn’t really muster the time and power to pick up the pace again. I tried. I failed. There were (and still are) quite a few distractions, either. Prominently an engaging software development project.

The main problem might have been, that I put to much pressure on myself. I was following advises from the blogging gurus. Like: make a schedule and keep to it; create a series; be predictive for your audience; it’s all about continuity and expectations.

I guess, that was simply to much. To much of a burden.

There is another aspect: my character. It forbids to become really good at anything. I can become quite good, even very good (in some cases and with enough force). But not perfect, no specialist, no guru. I’m a generalist par excellence. And that really bothers me at times.

As soon as something becomes too detached from the reality around me – no obvious connection to the lives of others, no impact, no sense – I flee. I stop doing it, just to see if there is some consequence. For at least an extended period of time.

So this is my last promise: I WON’T PROMISE ANYTHING.

2013 in review

The WordPress.com stats helper monkeys prepared a 2013 annual report for this blog.

Here’s an excerpt:

The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 38,000 times in 2013. If it were a concert at Sydney Opera House, it would take about 14 sold-out performances for that many people to see it.

Click here to see the complete report.

Spomet: Meteor Full-Text Search in a Nutshell

I recently updated my little full-text search project Spomet. I made some major changes to it’s API. As a consequence I just finished updating my earlier tutorial post, that covers the creation of an example app using Spomet for search. While doing this I realized two things: this tutorial is really long and it’s too basic for anyone who has already some bonus-miles riding the Meteor.

So I decided, to write this post: An in-depth, concise and to the point guide on using Spomet. It should serve as the main documentation for Spomet.

Get the package

In case you are using Meteorite / Atmosphere you can add Spomet to your app with:

mrt add spomet

If you prefer using plain Meteor you can grab the package code on GitHub and place it in a suitable folder (e.g. packages/spomet). Don’t forget to add the package to Meteor afterwards (e.g. meteor add spomet).

Adding Documents to the Index

Before searching makes any sense, there have to be some documents in the index. To achieve this, there are two functions to choose from. The first is Spomet.add, the first parameter is a hash and the second a function:

Spomet.add
        text: 'the text that should be found'
        type: 'post'
        base: someRefId
        path: 'description'
    ,
        (result) ->
            # the result is the hash from above 
            # substituted with omitted default values 
            # and the version number given

Spomet.add, well big surpriseadds a document to the index. The hashkeys type and path are optional and will be substituted with default values (‘default’, ‘/’) in case they are absent. The final function-parameter is optional as well. In case it is present it will be called when the document was successfully inserted with the initial document hash as parameter, extended with the version number used and eventual document parameter substitutions.

One thing to keep in mind though, the document gets added even if a document with the same identifying parameters (type, base, path) already exists. Internally there is a version number as a final part of the documents ID. This number gets increased. To actually know what version number was used you have to register the callback.

In case you don’t want to make different versions of the same document to be findable, there exists the second insertion-function: Spomet.replace. It adds the document and removes another occurrence of the same base document (identified by typebase and path). Spomet.replace takes optionally a second parameter, a version number. If this second parameter is present the specified version gets removed while the new document is added. If the parameter is absent the document with the biggest version number gets removed.

As the final parameter of Spomet.replace you might provide a callback. This one gets called with the document specs of the removed document, without the text parameter, though, and the same hash that get’s return from Spomet.add. Both are wrapped in a single object of the following form:

{
added:
    text: 'the text that should be found'
    type: 'post'
    base: someRefId
    path: 'description'
    version: 2
removed:
    type: 'post'
    base: someRefId
    path: 'description'
    version: 1
}

Spomet.replace adds the document even if there is nothing to remove.

Removing Documents from the Index

There is one function to remove documents from the index: Spomet.remove. I takes a hash parameter as well and removes all matching documents. Any combination is allowed. For example:

Spomet.remove
    path: 'description'

removes all documents with the path: description.

Spomet.remove accepts a callback as it’s final parameter as well. The callback function is called with an array containing the documents, that were removed. The documents in that array have the following form:

{
_id: "q6gQuRYayvnGttyyF"
base: "hiexeuv2kbyQ2Jnt8"
created: Mon Nov 04 2013 08:34:36 GMT+0100 (CET)
dlength: 22
docId: "custom-hiexeuv2kbyQ2Jnt8-custom-1"
indexTokens: Array[35]
mostCommonTermCount: 2
path: "custom"
text: "This is some test text"
type: "custom"
version: 1
}

Trigger Searches

You basically have two options. The first is to use the search-box that is provided by the package. The second is, use the search functionality but use your own implementation to trigger searches.

Use the Built-In Search Box

The built-in search-box provides autocompletion (using Bootstrap’s Typeahead) and uses the 1.000 most often frequented words, that are stored in the fullword index. There is a hash to configure, besides other things, the number of words exposed to the client. It’s accessible through Spomet.options from the server and has an attribute called keywordsCount. Alter it to the number of keywords you would like to expose to new clients.

To include the search-box you place the following code in your template of choice:

{{> spometSearch}}

This template call accepts a hash parameter to alter the way the textbox with it’s buttons is rendered. Currently the template tries to access fieldSizeClass, to get information on how wide the search-box should be rendered, and buttonText, to get the text that should be displayed on the search button.

Use the Plain Engine

You can use an arbitrary number of separate searches in your app and you are free to omit the provided search-box. You have to instantiate Spomet.Search and call find on the instance.

mySearch = new Spomet.Search
mySearch.find 'something'

To implement your own autocompletion you might want to access the collection Spomet.CommonTerms. The documents in this collection are primary ordered by number of indexed documents containing the keyword in question, secondary by the keywords length and have the following form:

{
_id: 'tRNE233c45DTrne'
token: 'meteor'
tlength: 6
documentsCount: 1
documents: [{docId: 'type-base-path-version', pos: 7}]
}

Like above, the number of keywords (tokens) is constrained by Spomet.options.keywordsCount.

Customization

In case you don’t want to search using all indexes you can explicitly set the indexes to be used during search:

mySearch.setIndexNames ['fullword', 'threegram']

The following indexes are currently implemented: fullwordthreegramwordgroup and custom.

Accessing The Results

Spomet.find doesn’t return anything. Instead, following the main Meteor paradigm, Spomet provides a reactive data source that updates itself and dependent user interface elements, while the search is underway. You access it with:

Spomet.defaultSearch.results()

in case you try to access the results from searching with the provided search-box. And with:

mySearch.results()

in case of a custom search. In either case the 20 documents with the highest score are published, sorted by score. You can alter this globally, by changing Spomet.options.resultsCount and Spomet.options.sort.

Besides this global options you can control the sort order, the number of results and the offset (for paging) on every Spomet.Search object. The methods for this are:

mySearch.setSort
    score: 1
mySearch.setLimit 50
mySearch.setOffset 50

The results() function returns a Meteor.Collection.Cursor object and the documents in this collection have the following form:

{
base: "cYeWPQ4s3uc8Aewmr"
type: "custom"
interim: false
phraseHash: "b32d73e56ec99bc5ec8f83871cde708a"
queried: Mon Nov 04 2013 09:33:37 GMT+0100 (CET)
score: 0.43085068485657296
subDocs: 
    "custom-1": 
        docId: "custom-cYeWPQ4s3uc8Aewmr-custom-1"
        path: "custom"
        version: 1
        score: 0.43085068485657296
        hits: [
            {indexName: "threegram", pos: 1, token: "not"},
            {indexName: "threegram", pos: 2, token: "oth"}]
}

The search results are grouped by base reference. In my experience this makes it easier to handle the display of the documents that are referenced in the search. The subDocs hash points to the actual matching documents (the documents that where added to Spomet), with their score and the actual hits.

The hits array holds a reference to the index (indexName), with which it was found, the offset (pos) in the document and the actual matching (sub-) string (token).

The attributes base and type are the same as from adding documents. phraseHash is a MD5 hashed representation of the search query. Queried is simply a timestamp when the search was first issued.

Searches are cached to improve performance. This search cache is flushed as soon as there are documents added or removed from Spomet.

The interim flag signals, if this result was created by a plain lookup in the local fullword index (true) or if the result was processed (or is currently being processed) on the server (false).

Controlling Index Usage

You have seen above, that you can control the indexes that should be used while searching. Additionally you might want to disable some indexes globally (for searching and indexing). You can achieve this by calling the corresponding Meteor methods:

Meteor.call 'disableFullWordIndex'
Meteor.call 'disableThreeGramIndex'
Meteor.call 'disableCustomIndex'
Meteor.call 'disableWordGroupIndex'

There exist corresponding enable methods, in case you want to re-enable an index later.

Adding Your Own Index

The implementation of indexes is pretty straight forward, actually. I haven’t tested it, but it should be possible to include your own index pretty easily. The following gist shows the threegram index. The inline comments should help you to get along. Keep in mind, though, indexes are exclusively for the server-side.

Olimex’s Programming Challenge #30

Oops I did it again. Last Friday I participated again in Olimex’s weekend programming challenge. It was the 30th. This time we had to come up with a numbers converter, which should convert arabic to roman numbers. For example: providing 34 should result in the output XXXIV. My current hammer is Meteor, so I made a little client-side-only Meteor app that does the trick. You can see it in action here.

My initially imagined solution worked out straight, so I finished in less than an hour on Friday night. In retrospection, the hardest part was constraining the input to valid numbers. I haven’t done this before, but will need this soon in another project. Which made the hastle worthwhile, I guess. At first here is the template code. Actually nothing fancy here.

The following gist shows the logic:

I have an array of arrays that hold the roman symbols. Every sub-array represents the two valid symbols for each decimal place. One below 5 and one for 5 and above. In roman there is a different symbol for every 5 steps.

The function updateRoman does the actual conversion. I step through the provided number, represented as a string, from left to right using the strings length as the offset for the roman symbols array. The switch statement handles the different cases. Special are 4 and 9, in those cases the ‘next’ symbol is used and instead of the usual addition, a subtraction is used by placing the lower symbol on the right side of the ‘next’ one. For example IV for four instead of IIII.

As a result I set a session variable that stores the roman representation. The session variable gets read in the Template.main.roman helper, which in turn gets called from the template. Since session variables are reactive, the UI reflects automagically the new value.

The keydown handler is responsible for letting only valid characters (0-9) through. At first I check if the key code of the pressed key doesn’t represent a control key (the arrow keys for example). If it’s not a control key the default behaviour (e.g. putting the value in the textfield) gets suppressed.

Next I check if the pressed key represents a digit. If the string is empty zero is not allowed otherwise all digits get through. I append the character representing the current key to the textfield and trigger the conversion function.

That’s it.

Common Mistakes Developing With Meteor – Collections

I have found many blog posts, guides and tutorials displaying the shiny new thing called Meteor. In all it’s glory. Those little nasty things hidden beneath aren’t discussed in detail or are simply left out. Don’t get me wrong I’m a huge fan of Meteor. But after struggling (again) for many days, I decided to begin this little list of things that I have learned, that aren’t so nice or have to be kept in mind. Plus a strategy how to deal with those cases. The word anti-patterns comes to my mind, but it doesn’t really fit.

In this first post I will talk about Meteor’s Collections. In later posts I will discuss further aspects. Template’s is another big area of confusion and will be discussed next. As the need arises and my understanding deepens I will update this and the latter posts accordingly.

Dealing with Meteor Collections

Collections are pretty complex things. They aren’t serializable to EJSON. No transport across the wire. But why would anyone want to do such a stupid thing, anyway? Well. I did. When starting off with Meteor I was thinking: ‘Damn, I have defined this Collection at the client (server) how do I make it available on the server (client)’. Stupid me, right? Until recently, I found out, that this is not an uncommon question.

Accessing Collections

First of all, this is simpler than you think. Defining a collection this way:

MyCollection = new Meteor.Collection ‘mine’

is really sufficient in most cases. But put this line in a file which is executed on the server AND the client.

Beware to call:

new Meteor.Collection ‘name’

more than once for every string identifier (‘name’ in this case). Meteor will throw errors. Meaningful one’s this time.

To make the collection accessible from other files you simple prepend the @ sign (in CoffeeScript). When you are writing package code, don’t use the @ sign, instead export the Collection in package.js, e.g.:

api.export(‘MyCollection’)

That make it visible in your app code and the other files in your package.

Dealing With Dynamic Collections

Meaning, you are creating Collections on the fly. You usually don’t need this, but in case.

You can’t store them in the Session object. That results in weird behaviour. I accidently did this, as I tried to store an object that itself held a reference to a Collection. Weird behaviour and no decipherable error messages, sadly.

Simply use a hash or a plain object to hold the reference to the collection.

Of course you can not store a Collection in another Collection. You might store the id of a document that you are referencing as well as the name of the Collection where the document belongs in another Collection. And instantiate the Collection object as well as find and read the referenced document as needed. Like so:

Loading Initial Data

If you try to load initial data (fixtures) on the client, you might end up with two (or more) documents in the collection. I don’t actually understand why this is the case. But that’s the way it is. The following code won’t work!

Wrapping the creation part in Meteor.startup () -> doesn’t help either. With the second page load you get a second document. That sucks, right?

You can create the initial documents on the server. I guess that’s the preferred way to do this.

Nevertheless you can do it on the client, as well, but you need to leverage Meteor’s pub/sub mechanism to achieve the desired result. The following gist works and survives reloads.

Pub / Sub

We have touched Meteor’s publish / subscribe mechanism earlier and I won’t go very deep in all the possibilities Meteor offers. Just one thing to keep in mind: subscriptions provide a partial view of the documents stored in the published collections. For example (like shown above) you can narrow down the number of documents published in a given subscription to exactly one by selecting the documents by their _id property.

For an in-depth discussion you should check out Discover Meteor‘s chapters dealing with pub/sub.

Fraud Warning: audible.de (an Amazon company)

Recently, my sister in law gave me an audio book in English (‘Bones of the Lost: A Temperance Brennan Novel‘, by Kathy Reichs) as a present. I prefer to read usually. But I thought it might be quite nice, to actually hear, how some of the words I regularly throw around here, sound actually. The second thought, I can listen to it while commuting and it will make the drive less stressful and prevent me from speeding.

Well it was really good, the experience. The book was OK, though. I wont’t bother you with a review.

But eventually the book has ended. Two days with “Funkhaus Europa”, on the edge of speeding again, and I missed listening to the English language. Desperately. Yesterday yet another Amazon parcel arrived and held, besides the intended content, a coupon for one free audio book on audible.de (don’t know if they operate outside of Germany).

I did as advised on the coupon, learned that I just bought a subscription with a monthly fee of 9,95€ with the first month free and decided rather randomly for ‘Good Morning, Midnight‘ by Reginald Hill. I downloaded the ‘.aax’ file and imported it to iTunes. So far so good. To attain something my car could understand, I tried to burn a MP3 disc, which iTunes refused to do.

Some research let me to the conclusion, that you have to burn regular audio CDs, rip those to MP3 and then burn a MP3 disc from those files. To circumvent burning real CDs I bought Virtual CD-RW by Burning Thumb, which acts as a real CD burner and outputs CD images instead of real CDs. After creating and ripping ELEVEN CDs I finally had MP3s in my iTunes library. I burned those to two real MP3 CDs and they work in my car, like a breeze.

Well, a somewhat smelly breeze, since the audio quality is really bad. I thought that might have to do with ‘the process’. But it hasn’t. The quality of the original file isn’t better. It’s too muted and the sibilants are far too prominent. It really hurts in my ears.

OK.

Given all this (the stupid format, the clumsy handling to make it work in my car, the bad quality) I decided that audible.de isn’t going to work for me in the future. So I decided to cancel my subscription. They promised, that you can do this anytime and simply online.

On my accounts page the link to cancel the subscription was quite prominent. I clicked on it.

And was redirected to a page holding some recommendations. Only at the bottom of the page was the question asked: ‘Do you really want to cancel?‘ and beneath it, a drop-down box where I should select my reason for the cancelation.

Screen Shot 2013-10-24 at 08.46.07

But selecting anything didn’t result in any notable behaviour. At first I thought I was already done. But a second look at my account revealed, that my subscription was still active.

I clicked the ‘cancel’-link again, selected a reason, still nothing happens. Trying the same thing with Safari (I’m usually using Chrome) yields the same result. I thought, well this might be an error on the page. I opened the JS console and really found some error messages.

I examined the area beneath the drop-down field and found an invisible submit button.

Screen Shot 2013-10-24 at 07.20.59

After discovering this, I was really pissed. No normal internet user is able to cancel a subscription.

The invisible button was even clickable. Dragging the mouse over the area where the invisible button resides turns the mouse pointer into the ‘pointing hand’. I clicked it and was redirected to yet another page. There, I should rate the likeliness of recommending audible.de to anyone else. I fumed and selected: absolute unlikely. Beneath the likeliness drop-down box was a pretty large text field, where I was encouraged to enter stuff, that I ever wanted to tell them. I started to vent my disappointment. But was cut off after one and half lines.

What the fuck.

The text box simply didn’t let me enter more!!!

That’s basically the reason why I have written this. I will see if I can come up with any eMail address at audible.de and Amazon to let them know what I think about such business practices. Amazon should better think twice with which companies they partner up. Amazon’s reputation is declining with such things. Even more than bad labor conditions for their workers, IMHO.

Screen Shot 2013-10-24 at 07.23.31

Follow

Get every new post delivered to your Inbox.

Join 120 other followers