Process a File Line by Line in Emacs Lisp

Xah Lee, 2008-12-05

This page gives a example of how to use emacs lisp to process a file line by line. If you don't know elisp, first take a look at Emacs Lisp Basics.

The Problem

Summary

I need to process a file line by line. For each line, i need to parse it into 2 parts, stringA and stringB, and i need to create a file and name it stringA. The file's content will have stringA and stringB, and other predefined text.

The following is a example of a input file:

at_rot_target(integer tnum, rotation targetrot, rotation ourrot)
at_target(integer tnum, vector targetpos, vector ourpos)
attach(key id)

And after the job, there should files named “at_rot_target”, “at_target”, “attach”. The content of “at_rot_target” would be like this:

# --
at_rot_target(integer tnum, rotation targetrot, rotation ourrot)
{
$0
}

Detail

I'm writing a major mode for Linden Script Language (LSL). LSL is a scripting language used for the virtual world Second Life. It has few hundred functions, and each one has parameters that is unusual as compared to normal programing languages. So, i want a function template feature in my major mode, so that a LSL programer can automatically insert function templates and see what are the parameters for that function and can fill it out as he code.

There is a easy-to-use template system package for emacs, called YASnippet. (see: YASnippet tutorial.) So, i decided to use this instead of implementing my own template system.

With yasnippet, it uses a plain text for template definition. To define a template, you need to create a file. For example, in LSL there's a function named “collision” with this syntax “collision(integer num_detected) { ... }”. This means, i must have a file named “collision” in the template dir, and the file content must be like this:

# --
collision(integer num_detected)
{
$0
}

This means, when my mode xlsl-mode is on, and yasnippet minor mode is on, then user can type “collision” followed by a keyboard shortcut for template completion, then the function form will be inserted and cursor will be placed between the braces.

I have prepared a file that is over 300 lines that are the LSL functions and parameters. For example, part of the file looks like this:

at_rot_target(integer tnum, rotation targetrot, rotation ourrot)
at_target(integer tnum, vector targetpos, vector ourpos)
attach(key id)
changed(integer change)
collision(integer num_detected)
collision_end(integer num_detected)
collision_start(integer num_detected)
control(key id, integer held, integer change)

(Save the above text in a file name it “xx_event_forms.txt” for later testing of elisp code.)

Now, the task is to parse this file, and for each line, create the file for it.

Solution

The task has these steps:

Open the file.
Read each line.
Parse the line into 2 parts. The first part is everything before the opening paren (call it stringA). The second part is the rest of the line (call it stringB).
Create a file and name it stringA.
Insert into the file the whole line, and other text such as “# --” and “{ $0 }”.

These are simple tasks. There are a lot ways to do this in elisp. We can for example grab the whole file's text, then use split-string by newline char to get a list of lines. Then we loop thru the line.

To read a whole file into a list of lines, you can use this code:

(defun read-lines (file)
  "Return a list of lines in FILE."
  (with-temp-buffer
    (insert-file-contents file)
    (split-string
     (buffer-string) "\n" t)
    ) )

The above method is familiar to perl, python programers.

Another way, is to simply open the file in a buffer, then move cursor one line at a time, each time grab the line and do what we need to do. In the following, we'll use this method.

First, we define few global vars.

;; input file
(setq inptuFile "xx_event_forms.txt")

;; other vars
(setq splitPos 0) ;; cursor position of split, for each line
(setq fname "")
(setq restLine "")
(setq moreLines t ) ;; whether there are more lines to parse

Now, we open the file, like this:

;; open the file
(find-file inptuFile)
(goto-char 1) ;; needed in case the file is already open.

Now, we loop thru the lines, like this:

(while moreLines
  (search-forward "(")

  (setq splitPos (1- (point)))
  (goto-char (line-beginning-position))
  (setq fname (buffer-substring-no-properties (point) splitPos))

  (goto-char (line-end-position))
  (setq restLine (buffer-substring-no-properties splitPos (point) ))

  ;; create the file
  (find-file fname)
  (insert "# --\n")
  (insert fname restLine "\n{\n$0\n}" )
  (save-buffer)
  (kill-buffer (current-buffer))

  (setq moreLines (= 0 (forward-line 1)))
)

In the above, we use search-forward to move cursor to the opening paren. Then, save the position to splitPos. Everything before that should be the template file name, so we save it in fname. Everything after that is restLine.

Now, we create the file fname using “(find-file fname)”, then, insert the content we want, save it, close it.

Lastly, we move cursor to the next line by “(forward-line 1)”. Note that if the cursor is at the last line, and when “forward-line” is unable to move forward, it will return a number indicating how many lines it failed to pass. So, normally it returns 0. If not, that means we are on the last line.

After we processed the lines, we just close the input buffer, like this:

(kill-buffer (current-buffer)) ;; close the input file

To test the above, first create a sample input file. Take the sample input lines above and save it as “xx_event_forms.txt” if you haven't done so already. Then, grab all the above lisp code and save in in a file “test_line_process.el”. (For convenience, download it here: elisp_process_lines.zip) Now, open the lisp file and call “eval-buffer”. Then all the template files will be created in the same dir.

Emacs is fantastic!

Was this page useful? If so, please do donate $3, thank you donors!

Home

About

Advertise

2008-12