Blogs | Alex Ho

Offline retrieval-augmented generation (RAG)

Sun, 11 Aug 2024 00:00:00 +0000

was my first experience with generative artificial intelligence (AI). I was impressed by its ability to make code suggestions and to generate unit tests. Translation between code comments and code was pretty seamless even the technology was based on ChatGPT-3.5 in 2022.

No matter it is GitHub Copilot or Chat GPT from OpenAI, it is available as a service which runs mostly as a black box to an end user. Although there were free tokens for anyone who wants to try it out, for more advanced usages like API calls to the service, a not insignificant amount of money would be charged. So I was not very interested in learning how does it work other than just learning enough to use it in my code editor for my $10 GitHub Copilot subscription.

Fast forward to early 2024, when I was messing with one of the tools named , I realised that there are options other than OpenAI offerings and the thing is named . Ollama allows users to run models locally and it has Docker support as well. I was intrigued by the idea of running a model locally and tried to run it on my laptop using Docker. Little that I know, at the time, that running a Docker image on Mac does not have GPU support and Ollama was running on CPU. Thus, even a small model like was super slow to spit out a response. Luckily, it did not take long for me to realise that I need to run it on bare metal on Mac to utilise GPU. I was truely amazed by the capabilities of the models that can be run locally on my machine.

Tools like works by talking to API of services like OpenAI. Similarly, Ollama has its own API initially but it later extended its API to add support for . The API makes tools can easily talk to the models via the API. Ollama is written in and uses to build the API.

As a software engineer, prompting the models via command line (or via a GUI like ) is definitely not enough for my appetite. Naturally, I started to think of building a semi-useful toy application and my idea is to build a question and answer CLI tool where it answers questions based on the markdown notes that I have locally on my machine.

To demonstrate the idea, here is one of the questions that I asked.

$ rag ask \
    -m llama3.1:8b \
    -n notes \
    -q "What is the best way to conduct an interview?"

And the response was

Based on the provided documentation, the best way to conduct an interview
involves a general approach that includes giving people a real problem to solve,
regardless of the medium of the exercise. This allows you to get a better sense
for how their brain works.

Here are some specific steps and tips mentioned in the documentation:

1. **Give people a real problem**: Provide candidates with a genuine challenge
   to tackle, which can help assess their thought process and problem-solving
skills.
2. **Use whiteboarding**: Whiteboarding can be useful for sketching out system
   designs or explaining thinking on a problem. It's essential to give people
   a chance to express themselves in the medium they're most comfortable with.
3. **Ask open-ended questions**: Try to ask questions that encourage candidates
   to share their thought process and approach, rather than just seeking
   a specific answer.
4. **Look for passion and curiosity**: Pay attention to whether candidates show
   genuine interest in the role, company, technology, and products. A lack of
   curiosity or defensiveness can be red flags.
5. **Provide feedback**: After the interview, don't hesitate to give feedback on
   what candidates did well. This can help them understand their strengths and
   weaknesses.

Additionally, the documentation suggests that you should:

* Reassure candidates that you're interested in how they approach problems, not
  just the answers.
* Ask questions that allow candidates to showcase themselves.
* Encourage candidates to explain complex concepts in simple terms.
* Look for passion and enthusiasm in a candidate's work.
* Use easy programming questions to filter out unsuitable candidates (e.g.,
  "Write a function that determines if a string starts with an upper-case letter
  A-Z").
* Leave time at the end of the interview to sell the company and job to the
  candidate.

By following these guidelines, you can conduct a more effective and informative
interview.

The response was based on and the model summarised the relevant parts of the markdown to answer the question pretty well.

Background

One of the ways to include customised data in a generated response is to pre-train the model with the data. Although this approach provides the best performace in terms of quality of response, it requires a lot of resources and time to train a new customised model. Another approach is to use where generation involves a step of retriving relevant information from a database of customised data and incorporating it into the generated response. This approach is more efficient in terms of resources and time as the model does not need to be re-trained with customised data which it can be changed frequently.

Preparing database of customised data

The steps required to load reference markdown files and to store in a database are outlined in this diagram.

To prepare the database of customised data, we load markdown files in a specified directory into memory. Along with the text of each markdown file, we also retrieve the metadata of the markdown file such as the filename. The metadata can be stored in the database the later steps.

The texts cannot be be directly stored into a database. One of the reasons is that the text is either too large to be used as embedding of a LLM or it contains too many topics which affects the quality of the response. To address this issue, text chunking is done before storing the text into the database. There are many text chunking techniques including the use of NLP to split text based on semantic meaning. However, I am no NLP expert and so I will use the simplest technique where text will be splitted into fixed-length chunks (1500 characters) and there will be a bit of overlap (300 characters) between chunks to avoid splitting up an important sentence. Each model has a limit of number of tokens to be used in an embedding. For example, model llama3.1 has a limit of 4096 token and model gemma2 has a limit of 3584 token. Therefore parameters used in chunking depends on the models to be used.

After splitting the text into chunks, a model is used to convert each chunk into a vector. The model used in this conversion is . There are many models can be used such conversion and it can be found on . The converted vectors along with its associated text and metadata are stored in a database. The database of choice is and it is a database for storing embedding vectors. Chroma stores text and metadata in a database and the associated vectors in its own indexed data structure.

The above steps can be summarised as the following code.

documents := retrieveDocuments(documentDirectoryPath)
splitter := createMarkdownSplitter(splitterChunkSize, splitterChunkOverlap)
splittedDocuments := splitDocuments(splitter, documents)
embedClient := ollama.New(
  ollama.WithModel(embeddingModelName),
)
embedder := embeddings.NewEmbedder(embedClient)
store := chroma.New(
  chroma.WithChromaURL(databaseURL),
  chroma.WithNameSpace(databaseName),
  chroma.WithEmbedder(embedder),
  chroma.WithDistanceFunction(types.COSINE),
)
for i := 0; i < len(splittedDocuments); i += VECTOR_STORE_BATCH_SIZE {
  end := i + VECTOR_STORE_BATCH_SIZE
  if end > len(splittedDocuments) {
    end = len(splittedDocuments)
  }
  store.AddDocuments(ctx, splittedDocuments[i:end])
}

where

ollama refers to package github.com/tmc/langchaingo/llms/ollama,
embeddings refers to package github.com/tmc/langchaingo/embeddings, and
chroma refers to package github.com/tmc/langchaingo/chroma.

An example of invoking the CLI application is as follows.

$ rag load -f ~/notes/ -n notes

The above command loads markdown files in the directory ~/notes/ into Chroma database with collection name as notes.

Generate answers to questions

The steps taken to generate answers to questions are outlined in this diagram.

The first step is to retrieve relevant documents from the vector database Chroma previously created. The retrieval is done by using model to convert the question into a vector and the vector is then used by Chroma to find the closest vectors in the database. Chroma will then return the associated text and metadata of the closest vectors.

To ensure the quality of the response, the retrieved documents are graded by a model to filter out irrelevant retrievals. The model used in this grading is the same model which will be used to generate an answer. The grading is done by asking the grader to give a binary score of yes or no to indicate whether the document is relevant to the question.

The following system prompt is used in the grader to specify the role we want the model (grader) to play.

You are a grader assessing relevance of a retrieved document to a user question.
If the document contains keywords related to the user question,
grade it as relevant. It does not need to be a stringent test.
The goal is to filter out erroneous retrievals.
Give a binary score 'yes' or 'no' score to indicate whether the document is
relevant to the question.
Provide the binary score as a JSON with a single key 'score' and no premable or
explanation.

User’s question and the retrieved document are then passed to the model via user prompt. The model will then generate a response in the form of a JSON which contains the binary score and can be parsed in the code.

After the grading, the remaining relevant documents are then used to generate the response for an answer to the question from user. The system prompt used in this step to specify the role of the model is as follows.

You are an assistant for question-answering tasks.
Use the following pieces of retrieved documentation to answer the question.
Please write in full sentences with correct spelling and punctuation. If it
makes sense, use lists.
If the documentation does not contain the answer, just respond that you are
unable to find an answer.
Explain the reasoning as well.

One of the major problems with generative AI is . To reduce the chance of this problem, statement If the documentation does not contain the answer, just respond that you are unable to find an answer. is used in the system prompt to let the model knows that it should not be too “creative” when there is no relevant context to answer a question. Thus, with the above system prompt set, the model will answer “I don’t know” if there is no relevant documents retrieved from Chroma database.

The following code summarises the above process of generating an answer to a user’s question.

embedder := getEmbedder(embeddingModelName)
store := chroma.New(
  chroma.WithChromaURL(databaseURL),
  chroma.WithNameSpace(databaseName),
  chroma.WithEmbedder(embedder),
)
retriever := vectorstores.ToRetriever(
  store,
  NUM_DOCUMENTS_TO_BE_USED,
  vectorstores.WithScoreThreshold(DEFAULT_SEARCH_SCORE_THRESHOLD),
)

documents := retriever.GetRelevantDocuments(ctx, askOpts.question)
if len(documents) == 0 {
  fmt.Println("No reference documents found")
  return nil
}

llm := ollama.New(
  ollama.WithModel(modelName),
)
relevantDocuments := []schema.Document{}
for _, doc := range documents {
  graderSystemPrompt := `
    You are a grader assessing relevance of a retrieved document to a user question.
    If the document contains keywords related to the user question,
    grade it as relevant. It does not need to be a stringent test.
    The goal is to filter out erroneous retrievals.
    Give a binary score 'yes' or 'no' score to indicate whether the document is relevant to the question.
    Provide the binary score as a JSON with a single key 'score' and no premable or explanation.`
  graderUserPrompt := fmt.Sprintf("Here is the retrieved document: %s \n\nHere is the user question: %s", doc.PageContent, question)
  gradingResponse, err := llm.GenerateContent(
    ctx,
    []llms.MessageContent{
      {
        Role:  llms.ChatMessageTypeSystem,
        Parts: []llms.ContentPart{llms.TextContent{Text: graderSystemPrompt}},
      },
      {
        Role:  llms.ChatMessageTypeHuman,
        Parts: []llms.ContentPart{llms.TextContent{Text: graderUserPrompt}},
      },
    },
  )
  if err != nil {
    return fmt.Errorf("unable to grade reference document: %w", err)
  }
  var graderResponse GraderResponse
  errParse := jsonhelper.ParseJSONString(gradingResponse.Choices[0].Content, &graderResponse)
  if errParse != nil {
    return fmt.Errorf("unable to parse grading response: %w", errParse)
  }
  if graderResponse.Score == "yes" {
    relevantDocuments = append(relevantDocuments, doc)
  }
}

if len(relevantDocuments) == 0 {
  fmt.Println("No relevant documents found")
  return nil
}

fmt.Printf("About to answer your question using %d relevant document sections...\n\n\n", len(relevantDocuments))

systemPrompt := `
  You are an assistant for question-answering tasks.
  Use the following pieces of retrieved documentation to answer the question.
  Please write in full sentences with correct spelling and punctuation. if it makes sense use lists.
  If the documentation doen't contain the answer, just respond that you are unable to find an answer.
    Explain the reasoning as well.`
relevantTexts := make([]string, 0, len(relevantDocuments))
for _, doc := range relevantDocuments {
  relevantTexts = append(relevantTexts, doc.PageContent)
}
userPrompt := fmt.Sprintf("Documentation: %s \n\nQuestion: %s \n\nAnswer: ", strings.Join(relevantTexts, " ; "), question)
llm.GenerateContent(
  ctx,
  []llms.MessageContent{
    {
      Role: llms.ChatMessageTypeSystem,
      Parts: []llms.ContentPart{llms.TextContent{Text: systemPrompt}},
    },
    {
      Role: llms.ChatMessageTypeHuman,
      Parts: []llms.ContentPart{llms.TextContent{Text: userPrompt}},
    },
  },
  llms.WithStreamingFunc(func(_ context.Context, chunk []byte) error {
    fmt.Print(string(chunk))
    return nil
  }),
)

where

embeddingModelName is set as nomic-embed-text,
vectorstores refers to package github.com/tmc/langchaingo/embeddings
ollama refers to package github.com/tmc/langchaingo/llms/ollama
llms refers to package github.com/tmc/langchaingo/llms

The command built using the code above can be demonstrated by the following.

$ rag ask \
    -m llama3.1:8b \
    -n notes \
    -q "What is the best way to conduct an interview?"

where llama3.1:8b is the model used to generate an answer via Ollama API and notes is the collection of documents stored in Chroma database.

The End

As you can see, this CLI application allows me to ask questions against my latest version of notes without training any new models and using models running offline on my laptop. When there is a new version of notes, all I need to do is to rebuild the Chroma vector database which takes about a minute for 200 markdown files on a laptop.

This is only my first exploration into generative AI. Things like can be build using . Hopefully, I will be able to spend time and build more stuff in the near future. I will share my journey in my future posts.

Happy coding!

Code of the CLI can be found on .

OAuth and PKCE

Thu, 13 Jul 2023 16:00:00 +0800

is a common mechanism used to communicate with APIs and its Authorization Code Grant is the most-used grant type. In writing a CLI application to talk to these APIs, the interactions look like the following.

To improve security, many API providers using Authorization Code Grant requires (Proof Key for Code Exchange, pronounced as “pixy”). This post is about how I write CLI clients implementing PKCE. But before diving into this, let’s go through what PKCE does first.

The Attack

From , the potential attack which PKCE helps to mitigate is

In this attack, the attacker intercepts the authorization code returned from the authorization endpoint within a communication path not protected by Transport Layer Security (TLS), such as inter- application communication within the client’s operating system.

The conditions for this attack to succeed are

attacker manages to register a malicious application on the client device and registers a custom URI scheme that is also used by another application. The operating systems must allow a custom URI scheme to be registered by multiple applications.
authorization code grant is used
attacker has access to client_id and client_secret
and either
- attacker (via the installed application) is able to observe only the responses from the authorization endpoint
  - this can be mitigated by code_challenge_method
- attacker is able to observe requests (in addition to responses) to the authorization endpoint. The attacker is, however, not able to act as a man in the middle due to TLS connection. This was caused by leaking http log information in the OS.
  - To mitigate this, code_challenge_method value must be set either to S256

From the above description of attacking vectors, it should be obvious that PKCE is not a replacement of OAuth client authentication as PKCE requires client_id and client_secret in order to work. For a longer explanation, you can read from Scott Bardy.

PKCE mechanism

As described above, the aim of such attack is to steal the authorization code so that it can be used to pretend to be a client to ask for an access token. PKCE is to ensure the traffic to authorization server is made by the true client. The mechanism of PKCE is not difficult to understand.

The client first generate a random code verifier code_verifier.

The client then send an authorization request with code challenge code_challenge which is a transformed code verifier t(code_verifier).

To ensure token request is made by the same client, the client send a token request with code verifier code_verifier. Upon receiving a token request, the server applies the same transformation to verify if the transformed code verifier is the same as the one it received in the authorization request.

In practice, transformation t involves hashing such as SHA256.

Code verifier is only being used once and cannot be reused. Thus, there is no chance of the code verifier being replayed.

PKCE OAuth client implementation in Go

const lengthCodeVerifier = 64

// character ~ is not included as some of the server implementations
// do not support it
//
// rune is a type alias to int
// effective this is an array of int
// each integer represents a Unicode code point
var letterRunes = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._")

func generatePKCEVerifier() string {
  generator := rand.New(rand.NewSource(time.Now().UnixNano()))

  b := make([]rune, lengthCodeVerifier)
  for i := range b {
    b[i] = letterRunes[generator.Intn(len(letterRunes))]
  }
  return string(b)
}

func generatePKCEChallenge(verifier string) string {
  hash := sha256.Sum256([]byte(verifier))

  // see https://www.rfc-editor.org/rfc/rfc7636#appendix-A
  // for no-padding requirement
  return base64
    .URLEncoding
    .WithPadding(base64.NoPadding)
    .EncodeToString(hash[:])
}

The challenge can be sent in an authorization request as one of the parameters.

codeVerifier := generatePKCEVerifier()
codeChallenge := generatePKCEChallenge(codeVerifier)
oauth2.SetAuthURLParam("code_challenge", codeChallenge)

Both challenge and verifier should be stored on client side so that challenge can be verified in redirection request and verifier can be sent in the subsequent token request.

ctx = context.WithValue(ctx, codeVerifierContextKey, codeVerifier)
ctx = context.WithValue(ctx, codeChallengeContextKey, codeChallenge)

Upon redirection from authorization service with authorization code, the challenge previously sent will be included in the URL of the redirection and verification is required.

queryParts, _ := url.ParseQuery(r.URL.RawQuery)
codeChallenge := queryParts.Get("code_challenge")
expectedCodeChallenge := ctx.Value(codeChallengeContextKey).(string)

codeChallenge should match the previously stored expectedCodeChallenge and the same applies to code_challenge_method (not shown in code snippets). If the validations pass, the verifier can also be sent in a token request as one of the parameters.

oauth2.SetAuthURLParam(
  "code_verifier",
  ctx.Value(codeVerifierContextKey).(string),
)

For an actual implementation of verification of codeChallenge and sending codeVerifier in a token request, you can find it in function .

I have written a function using channels and the above getTokenHandler to make it easier for a CLI application to utilise the OAuth workflow.

The End

That is all about implementing PKCE on client side. As you can see, it is not difficult. Hopefully this helps you understand the mechanism and allows you to implement yours.

Happy coding!

Bubble Tea

Fri, 17 Feb 2023 16:00:00 +0800

My software development work is mostly done in a terminal. So I suppose it is natural that most of my tools are using a . Besides writing bash scripts, there are situations where I need to write CLI programs myself from time to time. In my day job of writing enterprise applications, I almost use C# and JavaScript exclusively. However, when it comes to writing CLI tools, Go is the language I preferred. There are many CLI frameworks written in Go but is the one I used most. It is also a framework used by many tools like kubectl.

cobra itself is a very powerful framework and, combining it with viper and pflag, it has pretty much all you need in terms of writing a command line program.

Using arguments or flags with a CLI program makes it easy to automate things. However, there are times that I will need to handle more complicated interactions between users and the program. These interactions can be done via a but cobra does not have any coverage on it. As you could probably guess by now, is the tool which can be integrated into cobra to allow building a TUI with such interactions.

is a TUI framework from and it does not contain any UI components. It can be used without UI components and developer will have to write the code responsible for rendering and behaviour. Luckily, as also provides a set of common UI components in repository and it can be used in programs.

CLI programs which I built with bubbletea usually start with cobra as not all workflows require a TUI (such as my-app login). Some of the commands requires a TUI and that is when code will be written to start a bubbletea program.

An example on command login with cobra.

func runLogin(_ *cobra.Command, _ []string) error {
	token, err := authhelper.GetToken()
	if err != nil {
		return err
	}
	fmt.Println("Login has been completed successfully")
	return nil
}

An example on command list (which is interactive) with cobra.

func runList(_ *cobra.Command, _ []string) error {
  config, err := getOAuthConfigurationFromViper()
  if err != nil {
    return err
  }
  client, err := api.NewClient(config.GetOAuthConfig())
  if err != nil {
    return err
  }
  items, err := client.ListItems()
  if err != nil {
    return err
  }

  columns := getItemColumns()
  rows := getRows(items)

  t := table.New(
    table.WithColumns(columns),
    table.WithRows(rows),
    table.WithFocused(true),
    table.WithHeight(displayRowCount),
  )

  t.SetStyles(getStyles())

  m := tableModel{
    apiClient: client,
    table:     t,
    data:      items,
    tableKeys: tableEventKeys,
    inputKeys: inputEventKeys,
    help:      help.New(),
  }

  if err := bubbleteahelper.SetupLogFile(logFilePath, logFilePrefix); err != nil {
    return err
  }

  if _, err := tea.NewProgram(m).Run(); err != nil {
    return err
  }
  return nil
}

As you can see in the example with command list, although there is a bit setup work to be done before starting bubbletea, it is no very involved and relatively easy. The above example retrieves data from an API and populate it into a table. We setup a table with column headers, data (items), number of rows to be displayed, key bindings (tableKeys and inputKeys). We also setup of bubbletea with a log file as we cannot print log lines into a terminal since bubbletea takes over the whole terminal screen most of the time. tea.NewProgram(m).Run() returns the final model (which contains states) being used in bubbletea but we are ignoring it in this example.

One of the nice things with bubbletea is that the architecture (or program flow) can be easily understood. It uses . The architecture is mostly used to generate HTML renderings and the data flow is shown below.

(illustration taken from )

To start a program tea.NewProgram(m), developer will have to supply a model and the model m implements the following interface.

type Model interface {
	// Init is the first function that will be called. It returns an optional
	// initial command. To not perform an initial command return nil.
	Init() Cmd

	// Update is called when a message is received. Use it to inspect messages
	// and, in response, update the model and/or send a command.
	Update(Msg) (Model, Cmd)

	// View renders the program's UI, which is just a string. The view is
	// rendered after every Update.
	View() string
}

We can ignore Init() and focus on Update() and View(). Update() processes input Msg from bubbletea framework (or “Elm Runtime” in the illustration). Msg is likely an event (such as a key event, a window resize event or a custom event representing a business event). Update() contains the logic to process Msg and determines the model Model and command Cmd to be returned. Model responsible for all things about a UI including states and behaviour. Therefore, Model returned from Update() represents the next UI to be presented to the user while Cmd is a function returning a Msg for the next UI (Model) to be processed in its Update(). Update() has access to Model and, thus, it is also responsible to update states in Model. View() will be invoked after Update() and it is responsible for drawing (or writing characters) to the terminal to create UI required.

An example on Model and its functions.

type dateInputModel struct {
	fieldDescription string
	itemName         string
	input            textinput.Model
	originalText     string
	err              error
	help             help.Model
	keys             inputKeyMap
	parent           tea.Model
}

func (m dateInputModel) Init() tea.Cmd { return nil }

// Update updates the states of the date input
func (m dateInputModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
	var cmd tea.Cmd
	var cmds []tea.Cmd

	switch msg := msg.(type) {
	case tea.WindowSizeMsg:
		m.input.SetValue(m.originalText)
		m.input.Focus()
	case tea.KeyMsg:
		switch {
		case key.Matches(msg, m.keys.SaveAndExit):
			cmds = append(cmds, m.patchItem())
		case key.Matches(msg, m.keys.Cancel):
			return m.parent, showTable(nil)
		}
	case recordUpdated:
		return m.parent, showTable(msg.records)
	case errored:
		m.err = msg.err
	}

	m.input, cmd = m.input.Update(msg)
	cmds = append(cmds, cmd)
	return m, tea.Batch(cmds...)
}

// View renders the date input view
func (m dateInputModel) View() string {
	title := fmt.Sprintf("%s of '%s'", m.fieldDescription, m.itemName)
	errorMessage := ""
	if m.err != nil {
		errorMessage = m.err.Error()
	}

	return fmt.Sprintf(
		"%s\n\n%s\n\n%s%s\n",
		title,
		m.input.View(),
		getErrorMessageSection(errorMessage),
		m.help.View(m.keys),
	)
}

As you may be aware dateInputModel from the example above, a Model can contain multiple bubbles UI components (textinput.Model and help.Model in the example above) to help with rendering UI in View().

In Update() in the example above, besides it processes business messages recordUpdated and errored, it also processes key presses via tea.KeyMsg. As you can see in the switch statement followed, we can define keys to be handled in the UI. The following example shows how keys can be defined.

import "github.com/charmbracelet/bubbles/key"

func (k inputKeyMap) ShortHelp() []key.Binding {
	return []key.Binding{k.SaveAndExit, k.Cancel}
}

func (k inputKeyMap) FullHelp() [][]key.Binding {
	return [][]key.Binding{k.ShortHelp()}
}

var inputEventKeys = inputKeyMap{
	SaveAndExit: key.NewBinding(
		key.WithKeys("ctrl+s"),
		key.WithHelp("ctrl+s", "save and exit"),
	),
	Cancel: key.NewBinding(
		key.WithKeys("esc", "ctrl+c"),
		key.WithHelp("esc/ctrl+c", "cancel"),
	),
}

type inputKeyMap struct {
	SaveAndExit key.Binding
	Cancel      key.Binding
}

Package key from bubbles provides functionalities required to define keys. This allows naming of key bindings and assigning multiple keys to the same binding. It is also a nice feature that this allows defining help message for a binding.

is one of the most useful UI components is dealing with data. It is easy to setup table by calling its SetRows() with []Row where Row is just []string. We can also have key bindings previous example as show below.

With key bindings, we can provide actions to a table row and it makes interactions with or changes to a row possible. If you want, it is also possible to configure a table to allow multiple row selection which allows bulk action on selected rows. Table has a viewport included and that means height could be set to a table and, if not all rows of data fit into the table, scrolling will be enabled and it is a very nice UX.

is an UI component handling a number of items well. Each item can have its name and description.

Filtering on items before selection/de-selection is also possible. Each item in a list has to satisfy Item interface includes FilterValue() returning a string so that the list can use that value for filtering.

type Item interface {
	// Filter value is the value we use when filtering against this item when
	// we're filtering the list.
	FilterValue() string
}

// an implementation example
func (i listItem) FilterValue() string { return i.title }

Toggling of an item in a multi-select list can be done by adding a DefaultDelegate. (The following example does not allow toggling when multi-select is not enabled.)

type listSelectModel struct {
	fieldDescription   string
	originalSelections []string
	selections         []string
	multiSelect        bool
	itemName           string
	list               list.Model
	help               help.Model
	itemKeys           listMultiItemKeyMap
}

type listMultiItemKeyMap struct {
	ToggleSelection key.Binding
}

func newListSelectModel(fieldDescription string, originalSelections []string, selections []string, multiSelect bool, item string) listSelectModel {
	delegate := newListItemDelegate(listMultiItemEventKeys, multiSelect)
	newList := list.New(getListItems(selections, originalSelections), delegate, 0, 0)
	newList.Title = fmt.Sprintf("%s of '%s'", fieldDescription, item)
	return listSelectModel{
		fieldDescription:   fieldDescription,
		originalSelections: originalSelections,
		selections:         selections,
		multiSelect:        multiSelect,
		itemName:           item,
		list:               newList,
		help:               help.New(),
		itemKeys:           listMultiItemEventKeys,
	}
}

func newListItemDelegate(keys listMultiItemKeyMap, multiSelect bool) list.DefaultDelegate {
	d := list.NewDefaultDelegate()

	if !multiSelect {
		return d
	}

	d.UpdateFunc = func(msg tea.Msg, m *list.Model) tea.Cmd {
		switch msg := msg.(type) {
		case tea.KeyMsg:
			switch {
			case key.Matches(msg, keys.ToggleSelection):
				if i, ok := m.SelectedItem().(listItem); ok {
					i.selected = !i.selected
					return m.SetItem(m.Index(), i)
				}
				return nil
			}
		}
		return nil
	}

	d.ShortHelpFunc = func() []key.Binding { return keys.ShortHelp() }
	d.FullHelpFunc = func() [][]key.Binding { return keys.FullHelp() }

	return d
}

There are which we still have not explored here. I hope this gives a rough idea of what bubbletea and bubbles could do in writing CLI programs. If you want to learn more about it or products from visit .

I hope I will have time later to showcase more usages of the libraries. Until then, happy coding!

gum

Sun, 12 Feb 2023 22:00:00 +0800

I have been a full-time vim user for 5 years. Naturally, I’m also a heavy command line user and, even more naturally, I automate most of my workflows using commands in bash (sorry zsh). Writing scripts to automate stuff is fun. But there are times that I want to do stuff interactively with the shell especially workflow involving a number of parameters. One of the ways to go about it is writing a Go program. However, it could be too time-consuming even if I am proficient with writing Go. All I need is to glue different commands together. fzf is super handy when it comes to selections and it fits the use case of tmux very well. When it comes to things like asking for user input, presenting lines of data in a table or applying styles or formatting to output, there are not many good tools can be used with shell. Luckily, from can solve most of the problems mentioned here. It is an executable written in Go (and it also means it can be used in all major operating systems). charm.sh has a number of CLI tools created but we will focus only on gum in this post.

gum has a number of commands. choose, filter and file pretty much cover most of the functionalities in fzf and it probably has a nicer interface in terms of aesthetics. The usage of gum filter is very similar to fzf.

sha=$(gcloud builds list --limit 20 --format json \
  | jq -r '.[] | .substitutions.SHORT_SHA' \
  | fzf)

gum filter equivalent

sha=$(gcloud builds list --limit 20 --format json \
  | jq -r '.[] | .substitutions.SHORT_SHA' \
  | gum filter --limit=1)

Another example with multiple selections.

fd | fzf -m | xargs nvim

gum filter equivalent

fd | gum filter --no-limit | xargs nvim

The usage of gum choose is very similar to gum filter but slightly different in UI where it is tailored towards a selection of a smaller number of items and it does not have filtering functionality.

gum file is similar to running fzf in command line without any piped input.

The above commands are nice but it did not make me jumping to use gum. When I need to handle user input, gum input is really nice and it is something I cannot avoid anymore.

priority=$(gum input --placeholder "Enter priority" --value="$priority")

The above examples prompts user to input a priority number with a default number (supplied via $priority).

gum input also has masks built-in so that you can handle input like passwords.

gum write works in a very similar way to gum input but it handles multi-line input.

gum table is a super nice tool to present CSV data as a table in command line.

az sql mi list \
  | jq -r '.[] | .resourceGroup + "," + .name' \
  | gum table -c "Resource Group,Instance" -w 30,50

If option -c is not specified, it uses the first line of CSV as column headers. Option -w allows user to specify column widths (in number of characters).

One of the nice things with gum table is that it returns the selected row in CSV format. This allows further parsing of the selected data with commands like awk.

gum confirm has a nice interface in asking user to confirm before executing a command. For example,

gum confirm "Update keys?" &&\
  gcloud compute project-info add-metadata --metadata-from-file=ssh-keys=$key_file

gum style and gum format provides a lot of tools in allowing users to specify the aesthetics of the output.

echo IP address changed from $(gum style --foreground=#99000 $old_ip) to \
  $(gum style --foreground=#009900 $new_ip) in \
  $(gum style --foreground=#999900 $rg/$nsg/$rule) :+1: \
  | gum format -t emoji

For more practical examples, you can refer to the following files of bash functions.

There is also a videos tutorial from charm.sh.

More thoughts on Mastodon

Sat, 11 Feb 2023 15:00:00 +0800

This post is more like a continuation of I wrote back in early November 2022. As of the time of writing, I have been on Mastodon ( ) for 3 months now. I’m mostly happy with my experience so far and there are many positives. Although I have not made the complete migration from Twitter at the moment, I foresee that moment will not be too far in the future.

There are two kinds of people I follow on Twitter; tech people and journalists. The migration of tech people I followed on Twitter migrates was much faster than I thought. By now, most of them has been landed on Mastodon already. There are very few journalists in Hong Kong made the jump yet. I guess that is partly due to the fact that some of the Mastodon instances (servers) are not very reliable yet. Another reason could be accounts of journalists involves a lot of private posts (or direct messages in Twitter terms) and the fact that administrators of a Mastodon instance can see those posts is quite a security and privacy concern. Although the problem can be solved by news agency setting up their own Mastodon instance, I’m not sure if any small agency would like to include this undertaking.

The feed of Mastodon is a feature itself comparing to Twitter. Firstly, there are no promoted posts (imagine the minutes saved in a day…). Secondly, the feed is chronological and it means there is no infinite scrolling where posts are “suggested” by Twitter. This avoids biases from Twitter’s suggestions, or reading a viral topic, and, as many of us suggested, “dumb scrolling”. Another feature is that Mastodon has a “local” feed. What that feed composes of are posts from people on the same Mastodon instance. This greatly helps building a community (and another reason choosing an instance to join is an important step).

In relation to feed, a part of federated ActivityPub is also a feature not being utilised but I think it will be very popular in the future. is a Twitter-like implementation on ActivityPub. is a Instagram-like implementation on ActivityPub. Since both implementation speaks the same protocol (or “language”), users of Mastodon can read posts from a Pixelfed should it follows. That allows merging of multiple implementations and makes one big social platform.

APIs of Mastodon is much more powerful than Twitter. As a result, there are already many implementations of CLI tools. The one that I used often these days is (and it is built in Python). This allows me to browse social media without opening a browser.

You may not be able to get verified (blue tick) as a person on Mastodon. But you can definitely do it for the links you put onto your Mastodon profile. For the details, feel free to refer to .

Twitter was having a good balance between making it free for user and getting enough revenue. With recent chaos ( , ), I think users of Twitter will become part of the product and will not being served well anymore. Thus, I do think it is time to start considering “jumping the ship”.

Another social media app - Mastodon

Mon, 07 Nov 2022 19:00:00 +0000

Twitter and Facebook were my go-to social media apps back 10 years ago. I started using Facebook 2008 (yes, I was late to the party) and I connected to most of my friends there due to its popularity in Hong Kong. In 2009, via tech conferences (most of it were related to Docker), I learned that there is a thing call Twitter. Through Twitter, I learn a lot of stuff about Docker, later Kubernetes, and things in technology space in general. Given the way that Facebook treats user data, I tried to stay away from it since 2017. I rarely log onto it unless I had to (like contacting a friend who I only know on Facebook). Twitter had been the only social media app that I used since then. Twitter feed was batter than Facebook as there was less advertisements (back then) and the feed was mostly chronological (back then). I could get a lot of useful information from it and it does not serve too much noises.

Unfortunately, as with all the things in the last few years in Hong Kong and abroad, nothing is guaranteed forever. The way that was harsh but you may argue that it was a business decision. To continue to use Twitter, it is kind of endorsing the way Twitter will do things in the future in the name of free speech. Fortunately, there is a not-so-bad alternative for it and it is named .

How does Mastodon works?

It is federated and it is different from Twitter in terms of storing and serving the feeds. With Mastodon, there is no one company store and serving all the data. In fact, if you want to go through the pain of setting up a Mastodon instance, you can store and serve your own feed data. To understand how this federated network works, you can think of how email works. Creating a toot (or a tweet in old money) is like sending out an email. You will need to sign up with an email service or you create your own email server. Mastodon works the same way; you sign up for a Mastodon instance (server) or you create your own. When a toot is created, it will be stored and served by the server you signed up with.

There are potentially millions, if not hundreds of millions, of Mastodon instances and there is no way one can sign up for all of them to read the feed of other users. This problem is solved by which Mastodon uses. You can search for a handle (user) from your instance and the instance will search itself and other instances for that handle. Mastodon is able to discover each other through ActivityPub protocol. Thus, you can discover and follow others using ActivityPub and the protocol takes care of the data exchange between Mastodon instances. When a toot is created, it will be cached in the Mastodon instances which has users following the creator of the toot.

Good and bad

Mastodon is open-sourced and you can check its . This is good thing in terms of general security. For an average user, it also means there will be less “unwanted” features to be introduced. At the moment, there are no advertisements in the feeds which is a very good thing to me. The feed is also in chronological order which could save me a lot of time and ease my worry of missing something (and you know that feeling…).

Choosing a Mastodon instance to sign up with is important. It is partly because your data will be stored on that instance. Another important part is that moderation is not done by Mastodon but human moderators of your instance. Thus, each instance can have its own rules in terms of moderation and one instance can ban another instance. Another catch is that you probably need to know if your instance can scale up to the load to serve all users of it.

Search function is another good feature of Mastodon. You cannot search for any text of a toot but you can search and follow a hashtag. This is a good feature as it gives less a chance for anyone who wants to introduce noise to a conversation.

To be honest, there is a lot of room for Mastodon to be improved on security side of things. At the moment, data-at-rest in a Mastodon instance are not encrypted. That means all toots, direct messages and even passwords are stored in plain text. Thus, if you sign up with an instance, you are effectively trusting the administrators for your data. This is also the reason where 2FA (second-factor authentication) should be setup. (Reference: )

People using Mastodon are mostly in technology space at the moment. Thus, I guess it will take some for people to switch to use Mastodon. If you want to find out popular users (handles) which have made the move, you can probably search for #TwitterMigration and #followfriday.

There are also tools to help the migration from Twitter.

Future

Like my Facebook account, I will probably keep it but not log onto it unless I have to. My social media app to use will be Mastodon in the near future and you can find me with handle @alexhokl@hachyderm.io.

Thanks

Thanks to for spending time and money to set up a Mastodon instance. ( )

Git rebase with dependent branches

Sat, 08 Oct 2022 16:00:00 +0000

git rebase is always an important part of my workflow. As a maintainer of repositories at work, browsing commit history of a major part of my work. Thus, keeping the history “clean” is important. By “clean”, it means

code can be compiled and unit tests passes, and
code changes of a commit is complete and can be understood as a whole (not necessarily a complete feature but completeness on code level)

During active development in a feature branch, I prefer to commit small and often. This creates a lot of commits which it useful for me to understand the trail and to change the ordering of the commits. However, these small commits are probably only useful during active development but not so much when debugging the same code a few months later. Thus, my practice is always to rebase (in other words, re-writing the history of) the feature branch before creating a pull request.

Another reason for using rebase is to avoid any merge commits in a feature branch. I understand this controversial and it also depends on team’s policy. Personally, I do not see any use cases of having merge commits in a feature branch and I think it is only acceptable in main (or master). Thus, I will have a practice to rebase a feature branch before merging it into main.

There are times when development of a feature feature-b depends on another feature feature-b which is also in development. In this case, I would branch out from feature-a to create feature-b.

This works fine until I want to patch one of the commits originated from feature-a.

In such case, I will probably have to checkout feature-a, create a fixup commit, rebase feature-a to squash the fixup commit, checkout feature-b again, and rebase it against feature-a. The following shows the trail of commands involved. This is pretty involved.

git checkout feature-a
vim code.cs
git add code.cs
git commit --fixup feature-a~
git rebase --autosquash -i master
git checkout feature-b
git rebase -i feature-a

Thanks to , such patch can be done directly on feature-b. The trail of commands look like the following.

vim code.cs
git add code.cs
git commit --fixup feature-a~
git rebase --autosquash --update-refs -i master

Since feature-b has a reference of feature-a, option --update-refs will be able to help rebasing feature-a as well. This is very useful command when dealing with multiple development branches and it could save me a lot of time!

Hope this helps you as well.

vim

Mon, 31 Jan 2022 16:00:00 +0000

I have been actively using vim (technically ) for development at work and at home for almost 4 years (and was switching between between vim and atom the year before). I like it mainly because it allows me to use modal editing which is faster for me compared to or . Another reason is that almost all servers has either vi or vim available. One more reason is that I am pretty bad at using point-and-click interfaces and I would like to stick to command line tools as much as possible. Perhaps the most important reason, it looks cool to code in a terminal, isn’t it?

The learning curve of vim is pretty steep at the beginning. Luckily, I got myself a little head start by using . I then forked it and made . The way that I start learning vim is by going through and try to understand the mappings involved and try it out in vim. In retrospect, this is probably not the best way to start and I should have start with vimtutor and go through the basics instead. As many people have already suggested, I did not try to use it at work at the beginning as this would affect my productivity too much. After trying it out for 8-10 weekends with my hobby projects, I felt more confident with vim and tried to use vim and Visual Studio Code at work. After 3 or 4 months, I was getting more used to vim and use it solely at work. During this process, one of the things I have done to help myself learning more about vim is to consume a large amount of YouTube videos where people demonstrate their skills with vim. Here is a list of my favourite channels.

Besides YouTube videos, I learned quite a lot by reading through sub-reddits of and .

It is amazing that I can still learn something new about vim everyday now although I have been using it for more than 4 years full-time. The rest of this post is not meant to be a tutorial but rather showing what vim can do with some of my configuration and I hope this can inspire someone to use vim as their editor.

Apart from configuration, there are plug-ins to make life easier to use vim. There are many plug-in managers available. I inherited Jess Frazelle’s way of doing it by using from Tim Pope. I like this way of managing it as it is purely git operations (although everyone else is using which is good as well). The rest of this post is based on and many of the key bindings (or mappings) are not the same as the built-in ones.

If the following is too long a read for you, feel free to jump to .

Essentials

To be able to know what mappings are available is important for learning so that we would know what we can try. :Maps (with help of plug-in ) does exactly this.

File searching is also available using ctrl+p. It is like file search in any other editors or IDEs. Behind the scene, it is using fzf and .

^ can be used to jump to the first character of the current line.

$ can be used to jump to the end of the current line.

V can be used to visually select a line. Sometimes, one may want to select the previous visual select by gv. The following example visually select a bullet point, copy it, move the cursor somewhere else and select the paragraph again.

[s and ]s can be used to jump to the previous and the next spelling mistake respectively. When the cursor is on the spelling mistake, z= can be used show a list of suggested correct spellings. The following example jumps to the previous spelling mistake and take the first suggestion to fix it by [s1z=.

% can be used to jump between square bracket or parentheses surrounding the current cursor.

[{ and ]} can be used to jump to the previous and the next curly brackets respectively.

:jumps shows the list of locations of previous jumps made. ctrl+o and ctrl+i jumps to the previous and the next position respectively.

Editing

S starts cursor in the current line with correct indentation. It also wipes out the current line as well.

[+space and ]+space can be used to add empty line above and below the current line respectively. The following example uses 2[space to add to empty lines above the cursor.

[e and ]e swap the current line with the line above and below respectively. The following example demonstrates [e.

With cursor in parentheses of a method/function parameters, gs can be used to visually select a parameter and swap it. The following example selects the parameter to be swapped by using j and k, and the parameter is moved around by using h and l.

The same command can be used to swap a visually selected list as well.

Change operation can be pretty powerful. For example, 2ci( can be used to change all the text within two parentheses.

The above are some of the very basic stuff. I will cover more in the future blog posts.

Enjoy coding.

Hugo

Mon, 22 Apr 2019 19:45:03 +0800

It’s been a while since a write a blog post… It’s been 10 years. My passion on software development didn’t diminish in any way but I switched to making more than writing a blog post. Probably it was because it is pretty time consuming to write a post than to quickly added a few lines to my notes. The pros of writing a blog post is that I can document my experience on a problem I was trying to solve and, at the same time, I get a chance to consolidate my thoughts about it. More importantly, I always want to share the knowledge or experience to others as much as possible. So… here we go again…

My blog was originally hosted on . Since I have my containers hosted on the cloud 3 years ago, I decided to host it myself. I don’t really like to maintain WordPress and I like markdown very much. I ended up copying all the contents on Blogger and made it into markdown files. This workflow fits me very well as I always work in a terminal and is my default editor. Whenever I need to modify or add any content, I can work in a terminal and commit my changes to a git repository. To bridge the gap between markdown and HTML, I wrote some bash scripts to convert my markdown files to HTML pages using . With a bit of custom CSS, it was sort of acceptable to me in terms of look and feel. As one could imagine, quite a bit of directory structure design and bash script effort has to put in to make it working. Here is the Dockerfile I was using.

FROM golang:1.11.0-alpine as builder

RUN apk update && apk add git
RUN go get github.com/russross/blackfriday-tool

COPY google-analytics.html .
COPY create-blog.sh .
COPY create-pages.sh .
COPY markdowns pages/
COPY posts posts/

RUN ./create-blog.sh && ./create-pages.sh

FROM node:9.4.0-alpine as nodejs

RUN npm install -g clean-css-cli

COPY css/* ./

RUN cleancss -o base.min.css base.css
RUN cleancss -o retro.min.css retro.css

FROM nginx:1.13.8-alpine

RUN apk update && apk upgrade && apk add --no-cache --update curl

WORKDIR /usr/share/nginx/html

COPY manifest.json .
COPY js/sw.js .
COPY js/app.js ./js/
COPY images/favicon.ico .
COPY images/ ./images/
COPY move-pages.sh .
COPY --from=nodejs *.min.css ./css/
COPY --from=builder /go/pages/*.html ./pages/
COPY --from=builder /go/blog/ ./blog/
COPY html/index.html .

HEALTHCHECK --interval=5s --timeout=3s CMD \
  curl --fail http://localhost || exit 1

I heard about 6 months ago but, at the time, it did not sound something very promising as there is no clear path for me to generate a website quickly. Until… I met !

Academic allows me to generate a site in literally in a few seconds because they have a included. Academic is a Hugo theme created by George Cushen. After cloning the kickstarter from GitHub and pulling its theme in form of git submodule, one should have all the files needed. By running the following command, one should have a site serving on http://localhost:1313.

hugo server

The code to generated the stuff can be seen on the landing page can be found in directory content/home/ and it is very flexible as in one can freely add or remove any .md files to customise the site. Each .md file is made up of two parts, (or meta-data) and actual markdown content which is optional. For instance,

+++

title = "A Blog Post Title"
Date = "2019-04-22T18:00:00+0800"
# A front matter comment

+++

This is a **blog post**.

I have been through the process of writing custom bash scripts to generate content myself. Although a learn a lot from it, it is not something I wanted to maintain in a long run. Thus, Hugo plus Academic come as a very pleasant surprise to me as Hugo makes it very easy for me to generate HMTL content (it uses BlackFriday behind the scene as well) and Academic comes with themes which are good enough for my use case.

Hugo comes with a feature which makes it very easy to include multimedia content (actually all kinds of content) in markdown. The structure of a theme is easy enough to understand so that customisations is possible after working with it for a few hours.

I created two Hugo images (one for and the other for , see for usages) and my website and blog are generated by Hugo and served by Nginx. Here is the final image I used.

FROM alexhokl/hugo:latest AS build

COPY . .

USER root
RUN chown app.app -R .
USER app

RUN hugo

FROM nginx:1.15-alpine

WORKDIR /usr/share/nginx/html
COPY --from=build /home/app/site/public .

Hope this helps and more blog posts to come…

AJAX on Google App Engine

Sat, 04 Apr 2009 16:00:00 +0000

Continuing , I started to do something a little bit more than things in the tutorial. The tutorial basically shows the way to do traditional server-side web page processing. In terms of ASP.NET, the way is like coding all logics in a class implements IHttpHandler in the code-behind. As the reality is that we need dynamic pages in our web application, I tried to explore the ways to add some AJAX-behaviors to my web pages.

There isn’t any magic to use AJAX on App Engine (It just serves an incoming request, anyway). The difficult part comes in data serialisation. In an AJAX-enabled environment, you may have to pass a non-trivial amount of data to and from the cloud. With Python and App Engine, how do you parse the data from incoming request as well as prepare data in the response? Should you follow the trend, developers are now using JSON instead of old SOAP-compliant XML (maybe not even plain XML). What I want to do is retrieving some entities from datastore, extracting some of the fields, and encoding the fields into a JSON string.

I’m not giving a tutorial here. Honestly, I’ve never do any Python coding until last week. As a programmer coming from a C# background, the most difficult part is about the language and libraries available. So below is what I’ve done over this weekend.

List and Dictionary

These two are perhaps the most encountered containers. They are mutable. How to declare?

myList = []
myDictionary = {}

Where is my favourite statusMessage = abc == null ? “OK” : “Cancel” syntax?

statusMessage = ‘OK’ if abc == NONE else ‘Cancel’

The `get` handler

Here, Guest is a model in datastore. Then, I created a query named guests and a list named json. A for loop is written to iterate through the entities from datastore and extracted some of the fields to create a dictionary named g. By the end of iteration, “g” is appended to list json. For client side to retrieve the data correctly, I’ve to put the whole thing into a dictionary again and that’s why you see { 'guests': json } in the last line. The work of encoding Python object to a JSON string is done by a library called simplejson. Thus, you should have an import statement like this from django.utils import simplejson. Calling its dumps function is the last thing I have to do.

class GetGuestList(webapp.RequestHandler):
def get(self):
 guests = Guest.all().order('lastName')
 json = []
 for guest in guests:
   g = {
     'firstName': guest.firstName,
     'lastName': guest.lastName,
     'isVip': guest.isVip,
     }
   json.append(g)
 self.response.out.write(simplejson.dumps({'guests':json}))

The `post` handler

I didn’t try using JSON in posting data. But, if you want use it, I guess using simplejson.loads will be fine. Here, I used the ordinary techniques to post data back to server side.

class AddGuest(webapp.RequestHandler):
def post(self):
 isVip = self.request.get('isVip') == 'true'
 guest = Guest(salutation = self.request.get('salutation'),
             lastName = self.request.get('lastName'),
    firstName = self.request.get('firstName'),
    isVip = isVip)
 if users.get_current_user():
 guest.lastModifiedBy = users.get_current_user()
 guest.put()

The dispatcher

application = webapp.WSGIApplication(
                                  [('/', MainPage),
                                   ('/addGuest', AddGuest),
        ('/getGuestList', GetGuestList)],
                                   debug=True)

Here is how I did it.

Happy coding! :)

Initial Throughts on Google App Engine

Thu, 02 Apr 2009 16:00:00 +0000

I got some time this weekend to experience what cloud computing is all about. There are a few well-known “infrastructures” you can utilise at this moment, including Microsoft Azure, Amazon Elastic Compute Cloud (Amazon EC2) and Google App Engine. Honestly, I don’t have time to try Azure and EC2 yet. The reason I am attracted to App Engine is their “advertisements” in youtube. You can see that it looks very simple build a scalable web application from stretch. So, I gave a go this weekend.

No doubt at all. It is amazingly simple to startup. The only prerequisite is you must have python 2.5x installed. You can just download the SDK. For Windows users, the SDK is in form of an installer which helps you to make all the required path shortcuts (setting PATH environmental variable) ( ). For Mac users, there is even a UI that saves you a few console commands ( ). After downloading the SDK, I followed the tutorial to set up a simple guestbook application. The whole process only took me 30 minutes (including time to digest the python scripts and .yaml files) and surely it is painless.

As you may know, I’m coming from a .NET programming background. However, I found understanding python scripts is not all difficult. The hurdle to jump through to learn how to use App Engine is very low. For learning purposes, the quotas like CPU usage, datastore access, bandwidth usage, disk space, etc, that Google given is surely more than enough.

Before I go on to but some really useful applications, I’m now trying to understand the differences between “normal” server-client computing and cloud computing (actually, and between distributed computing and cloud computing as well). What I found so far is mainly about data store and caching. There are a few BIG words that I’ve come across about data store, like bigtable, entities, keys, entity groups, transactions. No matter what I have for my data, App Engine stores all data in a single table and that’s bigtable which allows good scalability. In the code, all a need to do is define a class which, in turn, defines what an entity is. Then calling its “put” method will save the entity into bigtable. Bigtable is not a relational database and it utilises time stamp property to maintain consistency. There are no table locks or row locks for write access. The write accesses are in a competitive environment and, if another transaction is writing to it, the transaction is rolled-back and try again. In bigtable, transaction is based on an entity group (a hierarchy of entities) and the assignments of entities to entity group is arbitrary. Thus, to allow efficient transactions, entity groups is to be designed very carefully to allow high throughput (basically, less disk writes). This is first difference I encountered so far. Relating to bigtable, there are a few interesting articles giving some insights in datas store designs, like (related to pattern and anti-pattern problem), , , etc. There is also a from Brett Slatkin talking about some advance techniques.

I’ll try to blog about caching in App Engine and more interesting link about this topic soon.

Happy coding! :)

Using embedded resources in embedded javascript files in ASP.NET

Tue, 31 Mar 2009 16:00:00 +0000

Embedding javascript files in an ASP.NET assembly is very convenient for deployment since no extra copying is required. This is especially useful when the javascript file is part of a control and is in an assembly which is consumed by other page developers.

It is sometimes hard to avoid putting some text strings in a javascript file. If these strings are coded into the javascript file, it poses a big problem to the one maintaining the code. Another problem may arise is that there is no way to localise (internationalisation, i18n) the strings efficiently. Fortunate enough, ASP.NET provides a way to embed resource files in assembly and retrieving its contents conveniently. Here, I will show you steps to accomplish this.

I assumed you know how to embed javascript files in an assembly and focus how these files access to the resources we are going to embed. You may read my previous post about to gain a favour of embedding javascript files.

What we want to have actually?

We want a javascript object that contains the resources we are going to embed.

Setup…

Okay. We know what we want to have. How do we get start? Don’t worry. We won’t start from ground zero. We got an “AJAX” button that it would say “Hello” to us when we click on it. Moreover, we got a page contains the button and a script manager. Here is the code of the button, the javascript stating its client-side behaviour, and the markups of the page containing the button. One thing to note is that the example below implements IScriptControl, but you don’t have to implement IScriptControl to use embedded resources.

And you have the solution setup looking like this.

If you have set things above up correctly, you should now see “Hello” popping when you click on the button.

Start…

Add a folder to the WebControls project and name it “ScriptResources”. Inside the folder, add a new item of type resource file and name it ScriptResource.resx. If you finish you should have solution structure like this.

Open the resource file and add resource with name as “LabelHello” and value as “Hello again!”. Once thing to notice is that if a resource file is added outside App_GlobalResources folder (ScriptResources, this time), it will automatically embedded into the assembly during compilation. You can verify this by checking is “Build Action” property.

Next thing you have to do is to add ScriptReference attribute to the HelloButton.cs like this.

The first argument of the attribute states which script file will use embedded resources. The second states which resource it uses. Note that first part of the string is the assembly name Alexhokl.WebControls. and the latter part ScriptResources.ScriptResources is the relative path of the resource file without extension resx. The third argument, as its name implies should be any arbitrary but unique name. Honestly, for “best practice”, I’d name it as Alexhokl.WebControls.Resources. But anyway, lets see how it works out in javascript file. In HelloButtonBehavior.js, modified the _onClick function to the following.

One last thing to do is to set attribute EnableScriptGlobalization of script manager to true. Compile and run the web application. You should now see the dialog box saying “Hello again!”.

Why?

Okay. It works. But what’s the magic? During compilation, the script manager acknowledged from the ScriptResource attribute that Alexhokl.WebControls.HelloButtonBehavior.js wants to use resource Alexhokl.WebControls.ScriptResources.ScriptResources and make it available as javascript object ArbitaryButUniqueName. Thus, the script manager just inject the ArbitaryButUniqueName object to the bottom of the script. Open the script file from browser using Firebug and you will see something like this at the bottom.

Bonus round

To test this implementation with internationalisation, we can add another resource file to the ScriptResource folder. Let us name it ScriptResource.de.resx and in the file add source with name as “LabelHello” and value as “Guten Tag!”. That’s it! Compile and run the web application. Now, when you click on the button, it should say “Guten Tag!” to you.

Again, how this is accomplished? You may feel more astonished when you look at the size of assembly. If you compare with the previously compiled one, you will find the size doesn’t change at all. The “magic” actually comes with an additional assembly in “de” folder. You should now see the Alexhokl.WebControls.resources.dll in the folder. This is the assembly responsible for script resources in German(de). Thus, when you deploy your assembly, make sure you deploy this localised folders as well; otherwise, the script manager will not be able to find the localised resources.

Summary

I hope you will now be able to use resource files with embedded script and globalising your scripts efficiently. In this post, I just outlined the steps without very detailed explanation. If you want to know what happen behind the sense, you should read an excellent article from MSDN magazine.

Hope this helps… :)

Creating ASP.NET controls with jQuery and MS AJAX (for MS AJAX dummies)

Sat, 28 Mar 2009 16:00:00 +0000

After reading an interesting tutorial from Janko about , I decided to wrap his javascript code, style sheets and HTML markups into an ASP.NET AJAX control. Wrapping them into a single control allows other page developers to use the control without worrying about the control’s details and keep focusing on the page’s overall functionality and layout. It is actually a simple task, but it can be very difficult for starters. I’ll do it in as much detail as possible and hope everyone will be able to follow the steps involved and know why things go this way.

Since I already have materials working without ASP.NET, the focus here is the techniques (or steps) required to wrap the materials in an ASP.NET AJAX control.

Okay. HTML markups first.

ASP.NET page developers usually consume assemblies containing HTML or web controls. Thus, I’m going to create a control living inside a normal .NET class library and named it as Alexhokl.WebControls. I will add a new class named SkypeLinkButton as well. The folder structure should look something like this after creation and addition.

Since the markup we want to render looks something like this:

The most convenient choice of class to inherit is ASP.NET LinkButton. As LinkButton is in namespace System.Web.UI.WebControls, I’ll have to add reference to System.Web.DLL and System.Web.Extensions.DLL. The code now should look something like this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

// added
using System.Web;
using System.Web.UI;
using System.Web.UI.HtmlControls;
using System.Web.UI.WebControls;

namespace Alexhokl.WebControls
{
  public class SkypeLinkButon : LinkButton
  {
  }
}

The button we want contains an image and a piece of text. To allow using custom image and text, we will use the inherited Text property and add an ImageUrl property. LinkButton is essentially responsible for rendering the anchor tag (<a>). To render the image tag and the text inside the anchor tag, we will override the CreateChildControls method. This method is invoked whenever the server needs to construct the control and maybe called more than once. Thus, besides adding the necessary controls to it, some housekeeping work is necessary.

public class SkypeLinkButon : LinkButton
{
  protected override void CreateChildControls()
  {
    this.Controls.Clear();
    base.CreateChildControls();

    HtmlImage img = new HtmlImage();
    img.Src = this.ImageUrl;
    img.Style[HtmlTextWriterStyle.Top] = "-4px";

    Literal literal = new Literal();
    literal.Text = this.Text;

    this.Controls.Add(img);
    this.Controls.Add(literal);

    this.ChildControlsCreated = true;
  }

  [UrlProperty]
  public string ImageUrl { get; set; }
}

The child controls are first cleared and the base method is called to create the necessary materials. To implement the image tag, we use HtmlImage; to implement the text, we use a literal control. We then add both controls to the control hierarchy by calling the Add method. Property ChildControlsCreated is set to true to tell the server that all the child controls are created successfully. This allows the server to skip another call to this method in the life cycle. (As you may notice in other implementations, EnsureChildControls are usually called everytime a control’s property is modified.)

Where is AJAX?

So far what we have done is just about ordinary techniques to create an ASP.NET control. The code above is good enough to render all the necessary HTML markups. What following up is embedding javascript and style sheet files in assembly. It should be reminded that embedding is not a must. But doing so will give us much convenience during deployment.

AJAX now? Not yet. Embed files first.

Now we will create two more files: one is SkypeLinkButton.css and SkypeLinkButtonBehavior.js. The file structure should look something like this after creation:

To embed the newly added files, right click on the file and select “Properties”. Changing the “Build Action” property from “Copy” to “Embedded resource” will allow the file to be embedded during compilation.

Done? Not yet. We need to enable the embedded files to allow the file to be served on web server. To do this, we use WebResource attribute.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

// added
using System.Web;
using System.Web.UI;
using System.Web.UI.HtmlControls;
using System.Web.UI.WebControls;

[assembly: WebResource("Alexhokl.WebControls.SkypeLinkButtonBehavior.js", "text/javascript", PerformSubstitution = true)]
[assembly: WebResource("Alexhokl.WebControls.SkypeLinkButton.css", "text/css", PerformSubstitution = true)]

namespace Alexhokl.WebControls
{
  public class SkypeLinkButon : LinkButton
  {
    protected override void CreateChildControls()

As you may notice, there are two arguments in the WebResource attribute. The second one is the familiar “content type” value. The first argument is the “web resource name” and is perhaps the trickiest part in embedding resource files. The argument is not defined arbitrarily and, if it is incorrectly defined, the web server will not be able to serve the file. The name is based on two parts: one is name of assembly and relative path of the file inside the assembly. Thus, the name of javascript file here can be seen as Alexhokl.WebControls. + SkypeLinkButtonBehavior.js. If the javascript file is placed inside folder named “Folder”, the web resource name will be Alexhokl.WebControls.Folder.SkypeLinkButtonBehavior.js.

MS AJAX starts here!

Now the files are able to be served on web server. Our next task is to request the files when page containing our control is loaded. What we want to do is:

Add script tags to load the javascript files
Add link tags to load the style sheet files
Add some javascript code to the bottom of the page to initialise the control on client side

From Janko’s demo, you can observe that “jumping” behaviour is added to the normal anchor link. Here, we want to create an object to encapsulate all the client-side behaviour. Fortunately, the MS AJAX framework provides us a very convenient way to accomplish this.

First, we need to make our SkypeLinkButton class implementing the IScriptControl interface. The interface declared methods that help mapping our javascript (client-side object) to our client-side element (which is the anchor element this time) and help registering a reference to the embedded javascript file. We will implement the two method like below.

public IEnumerable<Scriptdescriptor> GetScriptDescriptors()
{
  yield return new ScriptBehaviorDescriptor("Alexhokl.WebControls.SkypeLinkButtonBehavior", this.ClientID);
}

public IEnumerable<Scriptreference> GetScriptReferences()
{
  yield return new ScriptReference(this.Page.ClientScript.GetWebResourceUrl(this.GetType(), "Alexhokl.WebControls.SkypeLinkButtonBehavior.js"));
}

(yield keyword helps setting up a state machine to help setting up an enumerable object. You can choose return a list. Using yield to me is just syntactic sugar, although there is a bit of performance gain.) The above are just methods to help you setting up our javascript object, not creating them yet. We need to override life-cycle method’s OnPreRender and Render to create (should be “to render scripts to create”) the javascript object. Here is how we do it.

protected override void OnPreRender(EventArgs e)
{
  if (string.IsNullOrEmpty(this.ImageUrl))
    throw new Exception("Property ImageUrl cannot be null or empty.");

  this.scriptManager = ScriptManager.GetCurrent(this.Page);
  if (this.scriptManager == null)
    throw new HttpException("A ScriptManager control must exist on the current page.");

  this.scriptManager.RegisterScriptControl(this);

  // load jQuery from google CDN
  ClientScriptManager csm = this.Page.ClientScript;
  string protocol = HttpContext.Current.Request.IsSecureConnection ? "https" : "http";
  string jQueryScriptId = "jQueryScriptId";
  if (!csm.IsClientScriptIncludeRegistered(jQueryScriptId))
    csm.RegisterClientScriptInclude(jQueryScriptId,
      string.Format("{0}://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js", protocol));

  // load the embedded style sheet
  string cssUrl = this.Page.ClientScript.GetWebResourceUrl(this.GetType(), "Alexhokl.WebControls.SkypeLinkButton.css");
  HtmlHead head = this.Page.Header;
  if (head != null)
  {
    // need to make sure we only load the css 1 time
    if (head.FindControl(CreateValidID(cssUrl)) == null)
    {
      HtmlLink cssLink = new HtmlLink();
      cssLink.Href = cssUrl;
      cssLink.Attributes.Add("rel", "stylesheet");
      cssLink.Attributes.Add("type", "text/css");
      head.Controls.Add(cssLink);
    }
  }

  // set to use the css class we define in our embbed style sheet
  this.CssClass = string.Format("{0} skypebutton", this.CssClass);

  base.OnPreRender(e);
}

protected override void Render(HtmlTextWriter writer)
{
  this.scriptManager.RegisterScriptDescriptors(this);

  base.Render(writer);
}

private static string CreateValidID(string cssHref)
{
  string strBadCharacters = @" ./-\";
  foreach (var c in strBadCharacters)
    cssHref = cssHref.Replace(c, '_');
  return cssHref;
}

At the beginning of OnPreRender, some checkings are done to make sure we could render the control correctly. The scriptManager is a control required on a page using MS AJAX. However each page can only associated with one script manager. In line 10 of the above listing, our control is register to the page’s script manager so that the manager will be able to help creating our client-side behavior object. The remaining lines of code are to add the script tag to load the jQuery library from Google CDN and add the link tag to load the embedded style sheet file. One thing to notice is remember to call the overridden base method to make sure all the necessary things are prepared for rendering.

In Render method, we need to register script descriptors of our control to the script manager. What script descriptor does is to transfer server side properties or data to client side and what information is transferred is stated in the IScriptControl.GetScriptDescriptors method. For simplicity, I didn’t pass any data inside the method. But you can do so by creating a ScriptBehaviorDescriptor object and calling its AddProperty or AddElementProperty method to pass the required data, and then return this descriptor object (corresponding client-side javascript code has to be written to “collect” the data).

What is on client side?

You may notice the string Alexhokl.WebControls.SkypeLinkButtonBehavior in the IScriptControl.GetSriptDescriptors method. The string is actually the name of client side object we are going to implement. Okay, hold your breath. Here is the code skeleton.

///<reference name="MicrosoftAjax.js"/>

Type.registerNamespace('Alexhokl.WebControls');

Alexhokl.UI.WebControls.SkypeLinkButtonBehavior = function(element) {

  Alexhokl.WebControls.SkypeLinkButtonBehavior.initializeBase(this, [element]);

}

Alexhokl.WebControls.SkypeLinkButtonBehavior.prototype = {
  // lifecycle method
  initialize: function() {
    Alexhokl.WebControls.SkypeLinkButtonBehavior.callBaseMethod(this, 'initialize');

  },

  // lifecycle method
  dispose: function() {
    Alexhokl.WebControls.SkypeLinkButtonBehavior.callBaseMethod(this, 'dispose');
  }
}

Alexhokl.WebControls.SkypeLinkButtonBehavior.registerClass('Alexhokl.WebControls.SkypeLinkButtonBehavior', Sys.UI.Behavior);

if (typeof (Sys) !== 'undefined') Sys.Application.notifyScriptLoaded();

This is not that difficult to understand… believe me… but need a bit of time. The first line of content helps Visual Studio to locate the MS AJAX javascript library for the sake of Intellisense.(Yes, there is Intellisense) Many things here is trying the resemble things in .NET OOP model. The first function is basically a place for declaring “private variables”. For example, we may have this._myVariable = null; to declare a private variable _myVariable. The initialize method resembles the “constructor” of a class. Similarly, dispose method responsible for destructor’s actions. From Janko’s javascript code, all we need to do is to add code to the initialize method. After the addition, the whole thing should look like this.

///<reference name="MicrosoftAjax.js"/>

Type.registerNamespace('Alexhokl.WebControls');

Alexhokl.WebControls.SkypeLinkButtonBehavior = function(element) {

  Alexhokl.WebControls.SkypeLinkButtonBehavior.initializeBase(this, [element]);

}

SHG.SHINE3.Application.Web.UI.WebControls.SkypeLinkButtonBehavior.prototype = {
  // lifecycle method
  initialize: function() {
    Alexhokl.WebControls.SkypeLinkButtonBehavior.callBaseMethod(this, 'initialize');

    var element = this.get_element();
    $(element).hover(function() {
       $(this)'#' + $(this).attr('id') + ' img')
       // first jump
       .animate({ top: "-10px" }, 200).animate({ top: "-4px" }, 200)
       // second jump
       .animate({ top: "-7px" }, 100).animate({ top: "-4px" }, 100)
       // the last jump
       .animate({ top: "-6px" }, 100).animate({ top: "-4px" }, 100);
   });
},

  // lifecycle method
  dispose: function() {
    Alexhokl.WebControls.SkypeLinkButtonBehavior.callBaseMethod(this, 'dispose');
  }
}

Alexhokl.WebControls.SkypeLinkButtonBehavior.registerClass('Alexhokl.WebControls.SkypeLinkButtonBehavior', Sys.UI.Behavior);

if (typeof (Sys) !== 'undefined') Sys.Application.notifyScriptLoaded();

Added code is almost same as Janko’s code. Except we are using this.get_element() function to get the reference of the anchor element and then wrap it by jQuery’s $(). The function is available from MS AJAX framework. The function can return the anchor element because we have associated the client-side behavior object with the element in IScriptControl.GetScriptDescriptors method. To find the image for animation, children method is used.

Style sheet now…

Can we just copy Janko’s styles to our style sheet? Almost… Have a check on his styles you will find a background image will be loaded. Of course, you can code the URL and place the image accordingly. For the ease of deployment, we can embed the background image to the assembly as well. To do this, we first add the image to our project. Then, we select its “Build Action” property to “Embedded resource”. Next, we add WebResource attribute to again like below.

[assembly: WebResource("Alexhokl.WebControls.SkypeLinkButtonBehavior.js", "text/javascript", PerformSubstitution = true)]
[assembly: WebResource("Alexhokl.WebControls.SkypeLinkButton.css", "text/css", PerformSubstitution = true)]
[assembly: WebResource("Alexhokl.WebControls.bkg.png", "image/png")]

Here is our style sheet content.

.skypebutton {
  padding: 4px 10px 3px 25px;
  border: solid 1px #8AB134;
  position: relative;
  cursor: pointer;
  display: inline-block;
  background-image: url(<%= WebResource("Alexhokl.WebControls.bkg.png") %>);
  background-repeat: repeat-x;
  font-size: 11px;
  height: 16px;
  text-decoration: none;
  color: #40740D;
  -moz-border-radius-bottomleft: 5px;
  -moz-border-radius-bottomright: 5px;
  -moz-border-radius-topleft: 5px;
  -moz-border-radius-topright: 5px;
}
.skypebutton img {
  position: absolute;
  top: -4px;
  left: -12px;
  border: none;
}
.skypebutton:hover {
  color: #8AB134;
  text-decoration: none;
}

The WebResource tag inside will be replaced by a correct URL when the server parses this file. However, the server will do this kind of replacement only if the PerformSubstitution property is set to true for the WebResource attribute of the style sheet file.

Finished?

Yes. You can now deploy the compile assembly (Alexhokl.WebControls.DLL) or use the assembly to do your page development.

To learn more about MS AJAX, I strongly encourage you to set up a simple web application to see what is rendered in the HTML markups. There are a few thing you may observe:

link tag inside head tag


<link href="/WebResource.axd?d=YNaBRZ8x69XFlRxlm1R7jkGKSJLYaJ8WOtyPNxIc8I7PwXSdjS_SbOStsEQ9z3RGOFfrJJMYKZVMMuMb4HRXh4KnU8maEihGUCsmlUTOgd6GnXYbjVGceBEuvoHS6sJvjA-D6hYm3lBJFFzRKxDL_Q2&amp;amp;t=633739554426503190" rel="stylesheet" type="text/css" />

A link requesting WebResource.axd is rendered as a result of GetWebResourceUrl method is called inside OnPreRender method on server side.

script tags at the beginning of body tag


<script src='http://ajax.googleapis.com/ajax/libs/jqueryui/1.7.1/jquery-ui.min.js" type="text/javascript"></script>
<script src="/WebResource.axd?d=l53QigJd3KGiEVv-3KumAbiX189-7RprPkX3_4MKfr48x-zQ4acbiUL9qUInbHCWK4t6fBnLwHSUUUV5E62gBfH1B_kA8UWnXPpY7Eh0W6E1&amp;amp;t=ffffffffb65abaae" type="text/javascript"></script>

These script tags are added as a result of calling RegisterClientScriptInclude and RegisterScriptControl methods in OnPreRender method on server side.

anchor tag


<a onclick="onPayByCreditCard();" id="ctl01_mainContent_ctl00_ButtonByCreditCard" class=" skypebutton" href="javascript:__doPostBack('ctl01$mainContent$ctl00$ButtonByCreditCard','')">
<img src="/WebResource.axd?d=YNaBRZ8x69XFlRxlm1R7jkGKSJLYaJ8WOtyPNxIc8I7PwXSdjS_SbOStsEQ9z3RGOFfrJJMYKZVMMuMb4HRXh4KnU8maEihGUCsmlUTOgd7aZfH-0-s41XgMfAlQQRwRenIF4HCUQJ71zIzja2ZqQg2&amp;t=633739554426503190" style="top:-4px;" />
Pay by credit card
</a>

The element and its child elements are render as a result of code in CreateChildControls method.

behavior object is created by MS AJAX framework inside the script tag near the very bottom of the page (just before the body end tag)


<script type="text/javascript">
//<![CDATA[
Sys.Application.initialize();
Sys.Application.add_init(function() {
$create(Alexhokl.WebControls.SkypeLinkButtonBehavior, null, null, null, $get("ctl01_mainContent_ctl00_ButtonByCreditCard"));
});
//]]>
</script>

As you may notice from the script, MS AJAX framework initialises itself first. It then creates the behaviour object and associates it with the ID of the anchor element on client side.

Summary

This post just covers the basic steps to create an ASP.NET AJAX control. There are much more you can do with the control, such as passing server-side properties (using script descriptors) and globalisation (internationalisation) of the control (using ScriptResource). All of these tools are just helping you to render the correct HTML markups on client-side. Thus, I guess the best way to learn is always checking the client-side behaviour using tools like Firebug or IE Developer.

Hope this helps… :)

Javascript Syntax

Fri, 27 Mar 2009 16:00:00 +0000

Javascript is easy to write. I guess no one will disagree. Should you have some experiences in coding javascript, you may find coding in Javascript is error prone and is hard to debug. But why? It is partly due to its flexibility in syntax and it requires no compilation. However, if you give some care of the syntax, many errors can be avoided (you can’t avoid all those AJAX controls and RIAs development anyway). So, here we go…

1. No `==` or `!=` please…

Why not? Do you expect behaviours below?

'' == '0'            // false
0 == ''              // true
0 == '0'             // true
false == 'false'     // false
false == '0'         // true
false == undefined   // false
false == null        // false
null == undefined    // true
" \t\r\n " == 0      // true

But why? All these weird things happen because of the strange type conversion before the expressions are evaluated. To avoid these unexpected behaviours, always use triple equality operators (=== and !==) which return false in all of the above cases.

2. `null` and `undefined`

Can you tell the difference? If we talk like we are using a strong-typed language, null means “unassigned” whereas undefined means “no such variable”. Here are some examples to clarify:

var f1 = function() {
  var abc;
  if (abc === undefined) {
    return 1;
  }
  if (abc === null) {
    return 3;
  }
};

var f2 = function() {
  if (abc === null) {
    return 1;
  }
  if (abc === undefined) {
    return 3;
  }
}

var f3 = function() {
  var array = [1, 2];
  if (array[10] === null) {
    return 1;
  }
  if (array[10] === undefined) {
    return 3;
  }
}

All of the above functions returns 3.

3. Curly brackets

This is by far the craziest thing I have ever encountered in coding javascript. I guess not many of you may know. Which one is right?

function h1() {
  return {
    status: true
  };
};

function h2() {
  return
  {
    status: true
  };
}

What? Not all of them are right? Wait? There is right or wrong? They are just different styles, isn’t? Hell NO! Only function h1 is right, I tell you. h1 returns the expected object but h2 returns undefined. This is because javascript interpreter has a “nice” function named “automatic semi-colon insertion”. The results are listing one returns the expected object but h2 returns undefined. But why? Someone has a thorough explanation. I don’t repeat here. Please check the bottom of this post.

4. Try to avoid global variables

By global variables, what it means is a variable attached to window object. Since window object is available to everyone, it is why use of global variables is vulnerable to cross-site scripting attacks. Another reason is a weird implementation of Internet Explorer. IE7+ browsers attach every element with an ID a variable to window object. If you are using a global variable name same as an element’s ID, you will end up error message popping out. (for more detail see )

Did you notice all these things before? Don’t get red-faced. I just know all these a few days ago. This post is actually inspired by (should be “copied from”) a presentation by Douglas Crockford from Yahoo at Google TechTalk ( ) and is only about his beginning part. There are much more cool things about his presentation that worth noting. So try to make your javascript code better from today! And be sure to check out his .

Welcome

Sun, 26 Oct 2008 16:00:00 +0000

Hi all,

I’m alex. I’m going to blog about some programming or configuration tips in this blog. Here is a little bit about my background. I’m now working as a .NET programmer and on both WinForm and ASP.NET using C#. I like to build interactive AJAX controls. I’d describle myself as a component developer. Lately, i’ve bought an iPhone 3G and i’m now writing some simple programs. I’m on my way learning Objective-C and Cocoa.

I’ll post links to good tutorials and tips i found on the web and blog some tutorials that other didn’t talk about before.

Enjoy~

Blogs | Alex Ho

Offline retrieval-augmented generation (RAG)

Background

Preparing database of customised data

Generate answers to questions

The End

OAuth and PKCE

The Attack

PKCE mechanism

PKCE OAuth client implementation in Go

The End

Bubble Tea

gum

More thoughts on Mastodon

Another social media app - Mastodon

Legacy social media apps

How does Mastodon works?

Good and bad

Future

Links

Thanks

Git rebase with dependent branches

vim

Essentials

Navigation

Editing

Hugo

AJAX on Google App Engine

List and Dictionary

Where is my favourite statusMessage = abc == null ? “OK” : “Cancel” syntax?

The get handler

The post handler

The dispatcher

Initial Throughts on Google App Engine

Using embedded resources in embedded javascript files in ASP.NET

What we want to have actually?

Setup…

Start…

Why?

Bonus round

Summary

Creating ASP.NET controls with jQuery and MS AJAX (for MS AJAX dummies)

Okay. HTML markups first.

Where is AJAX?

AJAX now? Not yet. Embed files first.

MS AJAX starts here!

What is on client side?

Style sheet now…

Finished?

Summary

Javascript Syntax

1. No == or != please…

2. null and undefined

3. Curly brackets

4. Try to avoid global variables

Welcome

The `get` handler

The `post` handler

1. No `==` or `!=` please…

2. `null` and `undefined`