• Yorick Documentation On GitHub.
• Yorick Manual as a PDF.
• Yorick FAQ on Source Forge.

Really very very easy. Just do sudo apt-get install yorick! If you want to develop your own Yorick plugins then your will also need to do a sudo apt-get install yorick-dev.

apt-get install yorick yorick-dbg yorick-dev

For some of my little make jobs I want to be able to compile Yorick on two targets. On one target it is built from scratch so Yorick lives in one directory but on my Linux box it lives under /usr/bin and /usr/lib. Really the /usr/bin/yorick binary just points to /usr/lib/yorick/bin/yorick as Yorick tends to live under the one installation directory.

Usually to generate a plugin Makefile you'd just type...

yorick -batch make.i

... and get an auto generated Makefile with variables pointing to your Yorick installation written into the Makefile for you. For example, on my Ubuntu atp-gett'ed installation I get the following.

# these values filled in by "yorick -batch make.i"
Y_MAKEDIR=/usr/lib/yorick
Y_EXE=/usr/lib/yorick/bin/yorick
Y_EXE_PKGS=
Y_EXE_HOME=/usr/lib/yorick
Y_EXE_SITE=/usr/lib/yorick
Y_HOME_PKG=

One way to get the Yorick installation directory for a system that is independent of the particular Yorick installation is as follows.

YORICK_HOME="$(echo "print, Y_HOME" | yorick | grep '^".*yorick' | sed -e 's/"//g' | tr -d '\r' | sed -r 's/\/ *$//g')"
echo $YORICK_HOME

You can then substitute YORICK_HOME into the auto-generated Makefile so that it should work regardless who has the plugin and where their particular Yorick is installed (it might not be /usr/lib/yorick if they have installed in manually, for example).

Major Differences VS C Arrays

1. Yorick arrays start at index 1 but C arrays start at index 0.
2. Yorick arrays are column-major whereas C arrays are row-major.
   • This means that array[1][2] in C addresses row 1, column 2 (remember because C indices start at 0 this means the second row, third column). I.e., the C-syntax is array[ROW][COL]
   • But in Yorick, array(1,2), addresses column 1, row 2 (in C column 0, row 1 taking into account the start index of 1 instead of 0. I.e, the Yorick-syntax is array(COL, ROW).

What Is Row/Column Major?

For all languages, arrays are are contiguous blocks of items. Therefore, in memory we can store a 1D arrays as follows:

| ITEM[0] | ITEM[1] | ITEM[2] |  ...  | ITEM[N]
|         |         |         |       |
0 *sz     |         |         |       |
          1 *sz     |         |       |
                    2 *sz     |       |
                              3 * sz  :
                                      |
                                      N * sz

Here we store the first array element at the lowest memory address and increment upwards. You can see that each element is contiguous in memory. It is worth noting that the array is contiguous in terms of the language's model of the memory. For systems with an MMU this means virtual memory (which if the array crosses a page boundary it is only contiguous in virtual memory, not necessarily physical memory)

What happens when we try to store a 2D array? Consider the following array:

int someArray[2][3] = { { 1, 2, 3},
                        { 4, 5, 6} }

This is something we're very used to in C. We read this as an array with 2 rows and 3 columns. But how is this laid out in memory? It still has to be a contiguous chunk of memory but now there are two layout options. The C way is to lay out the array as follows:

| ITEM[0][0] | ITEM[0][1] | ITEM[0][2] | ITEM[1][0] | ITEM[1][1] | ...
|            |            |            |     ^^^    |          
0 *sz        |            |            | Note: new  |
             1 *sz        |            | row starts |
                          2 *sz        |            |
                                       3* sz        |
                                                    |
                                                    4 * sz

The array items are represented as a contiguous flow of values. The first row is laid out as it was for the 1D array, then the second row is appended to this block as if it were a 1D array and so on. Each row, therefore, starts at address row_index*3.

The C syntax specifies the fastest increasing index in memory from right to left. What does "fastest" mean? I'm using Munro speak here, and what he means is that, if we looked at the above memory layout, as we move one position from left to right the first dimension to increase is the "fastest", so in the C sense this is column.

Another way to think about this is considering an int p* = someArray pointer. As we increment this pointer by one (p++) it will traverse the columns in a row, which is what Munro calls the fastest increasing dimension, and then once all the columns in one row are iterated over it will move to the columns in the next row (so therefore the row is the second fastest increasing dimension) and so on.

We can see this in the above definition, int [2][3], where the right most dimension is [3] and represents the fastest increasing dimension in memory. The next dimension to the left is [2] and is the second fastest increasing dimension: rows.

This method of thinking about an array can be used for any number of dimensions. Consider int anotherArray[2][3][4][5]. We know that the fastest increasing dimension is 5, so if int *p = anotherArray then...

• *p     == anotherArray[0][0][0][0]
• *(p+1) == anotherArray[0][0][0][1]
• ...
• *(p+4) == anotherArray[0][0][0][4]

We also know that:

• *(p+a*5)     == anotherArray[0][0][a][0]
• *(p+b*4*5)   == anotherArray[0][b][0][0]
• *(p+c*3*4*5) == anotherArray[c][0][0][0]

So when we say row-major, we mean that, in terms of the syntax, the right most index is the fastest incrementing dimension and going left the speed of increment decreases.

Column-major therefore means, that, again in terms of the syntax, the right most index is the slowest incrementing dimensions and going left the speed of increment increases.

I keep saying "in terms of syntax" because as Yorick is implemented in C its physical array storage is C-like. It is only in the scripting language itself that column-major notation is used.

Adding More Dimensions: Row/Column Major Syntax in C VS Yorick

So, now moving onto ad 3D example... I'd normally visualise a 2x2x2 array as is shown to the right.

Given the description so far I would intuitively expect C to access the array using this syntax: array[DEPTH][ROW][COL]. I would expect Yorick to access the array using this syntax: array(COL, ROW, DEPTH).

In C: array[DEPTH][ROW][COL]

To be double sure, look at the noddy little program below:

#include 

int main(int argc, char *argv[])
{
   int array[2][2][2] = {
                           {
                              { 1, 2}, /* Depth, 0, row 0 */
                              { 3, 4}  /* Depth, 0, row 1 */
                           },
                           { 
                              { 5, 6}, /* Depth, 1, row 0 */
                              { 7, 8}  /* Depth, 0, row 1 */
                           }
                        };

   int r,c,d;
   for(d = 0; d < 2; ++d)
      for(r = 0; r < 2; ++r)
         for(c = 0; c < 2; ++c)
            printf("[d%i][r%i][c%i] == %i\n", d, r, c, array[d][r][c]);

   return 0;
}

It outputs the following...

[d0][r0][c0] == 1
[d0][r0][c1] == 2
[d0][r1][c0] == 3
[d0][r1][c1] == 4
[d1][r0][c0] == 5
[d1][r0][c1] == 6
[d1][r1][c0] == 7
[d1][r1][c1] == 8

So, in C, one have to visualise the inner most brace as being your slowest-incrementing dimension.

In Yorick: array(COL, ROW, DEPTH)

So, we can test how Yorick dimensions work. I would assume, given the Yorick syntax, Yorick would represent the array above using the syntax array(COL, ROW, DEPTH).

And again, to be sure, we make a little noddy program:

myarray = [
             [
                [ 1, 2], /* Depth, 0, row 0 */
                [ 3, 4]  /* Depth, 0, row 1 */
             ],
             [ 
                [ 5, 6], /* Depth, 1, row 0 */
                [ 7, 8]  /* Depth, 0, row 1 */
             ]
          ];

for (d=1; d<3; ++d)
   for (r=1; r<3; ++r)
      for (c=1; c<3; ++c)
         print, swrite(format="[d%i][r%i][c%i] == %i", d-1, r-1, c-1, myarray(c, r, d))
		

And, this outputs exactly the same as our little C program (the d/r/c-1 in the swrite() function is so that we print out the indexing from zero, as the C program would have done). This shows that our intuitive understanding of the syntax works for adding extra dimensions to arrays in Yorick is correct...

Array Indexing

a = [ [1,2,3], [4,5,6] ]

> dimsof(a)
[2,3,2]
 ^ ^ ^
 ^ ^ # rows
 ^ ^
 ^ # columns
 ^
 # dimensions

> a(1)
1
> a(2)
2
> a(3)
3
> a(4)
4
> a(5)
5
> a(6)
6

> a(2:3, 1:2)
[[2,3],[5,6]]

> a([2,1],[1,2])
[[2,1],[5,4]]

Slices Are Views Onto Arrays But Beware Of Array Copy

> a = [1,2,3,4,5,6]
> b = a(2:4)
> b
[2,3,4]
> b(:) = 111
> b
[111,111,111]
> a /*<-- LOOK: Unlike the NumPy example, a has not been affected! */
[1,2,3,4,5,6]

> a(2:4) = 111
> a
[1,111,111,111,5,6] /*< Aha! the slice is a view into the array */

When Arrays Are Not Copied By Value

> y = [1,2,3,4]
> func cheeseit(x) {
    x(1) = 9999 /*<-- NOTE: Will change array in caller's scope */
  }
> cheeseit(y)
> y /*<-- NOTE: y has been changed by function */
[9999,2,3,4]

> y = [1,2,3,4]
> func cheeseit(x) {
    x_local = x;       /*<-- x is copied to x_local, but copy is EXPENSIVE! */
    x_local(1) = 9999; /*<-- Changes to x_local will NOT affect array in caller's scope */
}
> cheeseit(y)
> y /*<-- NOTE: y is NOT changed by the function */
[1,2,3,4]

> a = save(q=[1,2,3,4])
> b = save(q=a.q) /* b's copy of a.q is a reference to a.q. unlike bare
> a.q              * array-to-array copy, which is by value, this copy is
[1,2,3,4]         * by reference! */
> b.q
[1,2,3,4]
> a.q(1)=999
> a.q
[999,2,3,4]
> b.q
[999,2,3,4]          /* LOOK! b.q must be a reference to a.q! */

Yorick, being a nicely vectorised language, lets you do mathematical operations between scalars and arrays, and arrays and arrays. The only condition is that the arrays are what is called "conformable".

Try thinking of it this way: if you can create the array B by joining together a load of A arrays then they will be conformable because the dimensions they share in common must be of equal length as B is just made up of many A's. The image below is meant to make that a little more clear...

The image is trying to show that, as the manual says, the shorter operand repeats its values for every dimension that is does not have. This is what is called broadcasting.

As a little side note, it appears that Yorick actually created the term broadcasting, to the extent that it inspired Python's NumPy broadcasting... wowzers!

We can see that because, for example, the 2D array is made up of two 1D arrays, the 1D array can be multiplied, added, etc with the 2D array by "expanding" into a 2D array by repeating itself in the second dimension so that it has the same shape. The same goes for 1D op 3D or 2D op 3D in the above example.

Let's see this in practice...

 Copyright (c) 2005.  The Regents of the University of California.
 All rights reserved.  Yorick 2.2.01 ready.  For help type 'help'
> a = [1,2]
> b = [a,a]
> c = [b,b,b]
> a
[1,2]
> b
[[1,2],[1,2]]
> c
[[[1,2],[1,2]],[[1,2],[1,2]],[[1,2],[1,2]]]
>
> a+b 
[[2,4],[2,4]]
>
> a+c
[[[2,4],[2,4]],[[2,4],[2,4]],[[2,4],[2,4]]]
> c+a
[[[2,4],[2,4]],[[2,4],[2,4]],[[2,4],[2,4]]]
>
> b+c
[[[2,4],[2,4]],[[2,4],[2,4]],[[2,4],[2,4]]]
> c+b
[[[2,4],[2,4]],[[2,4],[2,4]],[[2,4],[2,4]]]

As per our little image above... the arrays are conformable and so can be operands to the normal mathematical operations. BUT... the above does not give the whole story. "In common" can be more flexible. For example, we can do the following...

> aa = [1]
>
> dimsof(aa)
[1,1]
> dimsof(b)
[2,2,2]
>
> aa + b
[[2,3],[2,3]]
> aa + c
[[[2,3],[2,3]],[[2,3],[2,3]],[[2,3],[2,3]]]
> bb = [[1], [1]]
> bb+c
[[[2,3],[2,3]],[[2,3],[2,3]],[[2,3],[2,3]]]
		

Now, given the above definition, one might thing that the common dimension between aa and b does not have the same length, yet they are clearly conformable as far as the Yorick interpreter is concerned. What's going on?

The reason that this works is that Yorick will broadcast any unit-length dimension in addition to a missing final dimension.

Therefore, pictorially, we can see the following...

So far we seem to have the following definition...

One does have to take a little care in what is understood by "dimensions in common", however. This might appear obvious to you, but I had to do a double take...

> a = [1,2]
> dimsof(a)
[1,2]
> b = [[7,8,9,0],[6,5,4,3]]
> dimsof(b)
[2,4,2]
> a+b
ERROR (*main*) operands not conformable in binary +
WARNING source code unavailable (try dbdis function)

If we look at the dimensions with the following formatting we might be fooled into thinking that a and b share a common dimension of exactly the same length, and therefore should be conformable.

/* Caution: This is an INcorrect grouping of the dimsof result */
                +++
dim_a =    [ 1, |2| ]
dim_b = [ 2, 4, |2| ]
                +++

Remember that because Yorick is column major the dimsof array has the format [2, cols, rows]! Writing out the dimensions as I have above is quite misleading, and the mistake I at first made, because a has 2 columns and b has 2 rows and 4 columns. They are therefore NOT conformable. Remember to group your dimensions correctly:

/* This is the right way! */
             +++
dim_a = [ 1, |2| ]
dim_b = [ 2, |4|, 2 ]
             +++

Pseudo Index

The Yorick manual example starts with the outer product of two vectors. The following is taken from the linked-to Wikipedia article...

Assume that there are two column vectors $u$ and $v$.

$u = \begin{pmatrix} u1 \\ u2 \\ u3 \\ u4 \end{pmatrix}$ and $v = \begin{pmatrix} v1 \\ v2 \\ v3 \end{pmatrix}$

The outer product of the two column vectors is defined as follows...

$u \otimes v = uv^T = $ $\begin{pmatrix} u_1 \\ u_2 \\ u_3 \\ u_4 \\ \end{pmatrix}$ $\begin{pmatrix} v_1 & v_2 & v_3 \\ \end{pmatrix} = $ $\begin{pmatrix} u_1v_1 & u_1v_2 & u_1v_3 \\ u_2v_1 & u_2v_2 & u_2v_3 \\ u_3v_1 & u_3v_2 & u_3v_3 \\ u_4v_1 & u_4v_2 & u_4v_3 \\ \end{pmatrix}$

The matrix multiplication of those two vectors puzzled me for a little bit (I've only ever been used to multiplying matrices with dimension sizes greater than 1). It may be easier to think of it as follows...

$\begin{pmatrix} u_1 \\ u_2 \\ u_3 \\ u_4 \\ \end{pmatrix}$ $\begin{pmatrix} v_1 & v_2 & v_3 \\ \end{pmatrix} \equiv $ $\begin{pmatrix} u_1 & 0\\ u_2 & 0\\ u_3 & 0\\ u_4 & 0\\ \end{pmatrix}$ $\begin{pmatrix} v_1 & v_2 & v_3 \\ 0 & 0 & 0 \end{pmatrix}$

How do we do this using Yorick? All one dimension vectors in Yorick are column vectors (even if they are displayed textually as-if they are a row!). So can we just transpose() the $v$ vector?

> a * transpose(b)
ERROR (*main*) operands not conformable in binary *
WARNING source code unavailable (try dbdis function)
> transpose(b)
[100,200,300]
> dimsof(transpose(b))
[1,3]

Apparently not! transpose(b) didn't do what it might have... I thought it would turn the column vector into a row vector, but it doesn't. The reason for this is that a row vector is actually a 2D array, i.e., it has an extra dimension.

And this makes sense... as we saw in our section on adding extra dimensions to arrays and how to syntactically express this: we know a 1D vector is array(col) and a 2D vector is array(col, row). Thus, is makes sense that a row vector is a 2D array with one column!

So, to convert $v$ into a row vector we need to add an extra dimension...

To convert $v$ into a row vector we need to make each column element it's own row in a 2D vector. If $v$ has 3 columns, $v^T$ has 3 rows, where each row has only one element. To do this in Yorick, as said, we need an extra dimension... enter the pseudo index...

> b
[100,200,300]
> b(-,)
[[100],[200],[300]]
> b(,-)
[[100,200,300]]

The syntax (-,), really the symbol "-", is the pseudo index. "-" means "add an extra dimension here".

Munro calls the - sign, when used as an index, a pseudo-index because it actually inserts an additional dimension into the result[ing] array which was not present in the array being indexed. And, as we noted above, to go from a column vector to a row vector (i.e, transpose the column vector) we need to increase the dimensionality of our array.

So, based on this, we can say that b(-,) is saying add a row dimension to the column vector, or more generally add another most-slowly-increasing dimension to the existing vector. In the example above we get a 2D vector with 3 rows and 1 column.

How the extra dimension is added is both interesting and important. If b(-,) is saying add a row dimension, why does b(,-) == [[100],[200],[300]] and not [[100,200,300]]? They both have a row dimension added!

I think of it in the following way. Remembering that Yorick is column major, anything to the left of the "," in (,) indicates a faster dimension and anything to the right a slower dimension. Therefore, (-,) is saying add a faster dimension and (,-) is saying add a slower dimension... and this kinda works...

This generalises... suppose we can c a two dimensional array and we write c(-,).

> c = [[1,2,3],[4,5,6]]
> c
[[1,2,3],[4,5,6]]
> c(-,)
[[[1],[2],[3]],[[4],[5],[6]]]

/* 
 * c hand-formatted is...
 *    [
 *       [ 1,  2,  3],
 *       [ 4,  5,  6]
 *    ]    |   |   |
 *         |   |   |
 * c(-,) hand-formatted is...
 *    [    |   |   |
 *       [[1],[2],[3]],
 *       [[4],[5],[6]]
 *    ]
 */

To add the extra dimension, each column element becomes a row with one column, and therefore the rows become depths. More generally it is like we shift the dimensions up one... The fastest increasing dimension is changed so that each individual element inside it is now the fastest increasing dimension and all the containing dimensions become one slower! Ie., we've added a new fastest dimension.

Now we can look at b(,-) . Given the discussion so far it looks like we are saying add a column dimension. We have seen that b(,-) == [[100,200,300]]. And this also makes sense... we've added a new slowest dimension.

So back to the original example of outer product. To accomplish this in Yorick we need to do transpose(u*v(-,)) I.e., the transpose of v is v(-,):

> u = [1,2,3,4]
> v = [1, 10, 100]

> v(-,)
[[1],[10],[100]]

> u * v(-,) /* This won't be quite right! */
[ [  1,   2,   3,   4],
  [ 10,  20,  30,  40],
  [100, 200, 300, 400] ]

> transpose(u*v(-,)) /* YAY, now we're good! */
[ [1, 10, 100],
  [2, 20, 200],
  [3, 30, 300],
  [4, 40, 400] ]

Matrix Multiplication & Inner Product

Remember to keep in mind that Yorick is column-major so the indices are a(col, row)!

The concept of matrix multiplication, the type we learnt in school, is shown in the little animation below. For each row vector of the LHS matrix we take the dot product of that vector with each column vector from the RHS matrix to produce the result:

In Yorick the '+' sign labels an array index for use in the dot product (a.k.a. inner product or scalar product) and can be used to calculate vector inner products or matrix multiplications. Munro has generalised the "school-learnt" matrix multiplication concept so that we are not constrained as to the choice of row/column vectors we take from the LHS and RHS. For example, we might want to take the dot product of column vectors from the LHS and column vectors from the RHS.

For two 1D arrays, a and b, the operation is fairly straight forward. Writing a(+) and b(+) marks the only dimension in each array as being used for the inner product. Therefore, the calculation becomes $\Sigma a(i)b(i)$. Let's take a look at this pictorially, otherwise the 2D example melts my brain a little...

Now we move on to the more complex examples... as always, remember to keep in mind that Yorick is column-major, so the indices are M(col, row).

LHS column vectors and RHS row vectors

I feel like the above needs a little explanation before continuing with the others.

Munro describes the plus symbol as "marking an index for use in an inner product". But what exactly do we mean? It means that the index used is the one that we iterate over by summing the multiplication of each element with it's respective "partner" in the dot product. Thus a(, +) means iterate over the row indices in the dot product summation. If we are iterating over the row index, then the column index must remain constant, so we will be taking column vectors.

To understand what the + sign means we could think of it as meaning "selecting the dimension to be iterated over in the summation of the inner product", because what the above does is to select all dimensions not marked by a + sign, and then iterate over the marked dimension marked for the inner product.

So, if the + sign marks rows, we are NOT selecting row vectors and then using the row vector for the inner product. We ARE selecting column vectors and using the rows from each column vector in the inner product. The same is also happening with respect to b(+,): we are selecting rows and iterating over the columns.

I.e., for the + sign we select the vectors in the other dimensions and iterate over the marked dimension in those selected vectors.

So, now we can continue looking at the other possible ways to multiply out these square matrices...

LHS column vectors and RHS column vectors

LHS row vectors and RHS row vectors

LHS row vectors and RHS column vectors

We can see from the above that the index marked with the + sign is used for the inner product. Note that all these combinations are only possible because the matrices are square (for rectangular matrices, normal matrix multiplication shapes apply).

For example, in the pictures above, a(,+) marks the row index as being used for the inner product. I.e by selecting all rows in each column we create the vectors being used on the LHS of the dot product. Therefore we select a(1,:), a(2,:) all the way through to a(n,:).

Therefore, the indices marked with the + sign must have the same length.

Lets say dimsof(a) = (2, n, m) and dimsof(b) = (2, m, l). The process for c = a(,+)*b(+,) becomes:

for i in 1..n 
   v1 = a(i, :) // 1d vector, length m
   for j in 1..l
      v2 = b(:, j) // 1d vector, length m
      r(i,j)  = v1(+) * v2(+)

Normal or "School-Learnt" Matrix Multiplication vs Yorick

The only matrix multiplication I was ever taught was "normal" matrix multiplication:

$$\begin{pmatrix} a(1,1) & a(2,1)\\ a(1,2) & a(2,2)\\ \end{pmatrix} \begin{pmatrix} b(1,1) & b(2,1)\\ b(1,2) & b(2,2)\\ \end{pmatrix} = $$ $$\begin{pmatrix} a(1,1)b(1,1) + a(2,1)b(1,2) & a(1,1)b(2,1) + a(2,1)b(2,2)\\ a(1,2)b(1,1) + a(2,2)b(1,2) & a(1,2)b(2,1) + a(2,2)b(2,2)\\ \end{pmatrix} = $$ $$\begin{pmatrix} a(:,1)b(1,:) & a(:,1)b(2,:)\\ a(:,2)b(1,:) & a(:,2)b(2,:)\\ \end{pmatrix}$$

What you might notice is that the way the matrix multiplication works in Yorick, as described above, the index of the RHS changes most quickly. In "normal" matrix multiplication, it is the LHS index that is changing most quickly.

The figure below compares the closest operation we've seen so far with what I'd think of as a "normal" matrix multiplication...

Visually we can see that transpose(a(+,)*b(,+)) is equivalent to the "normal" matrix multiplication.

Another way to do this is to note that... $$\begin{pmatrix} a(:,1)b(1,:) & a(:,1)b(2,:)\\ a(:,2)b(1,:) & a(:,2)b(2,:)\\ \end{pmatrix} \equiv \begin{pmatrix} b(1,:)a(:,1) & b(2,:)a(:,1)\\ b(1,:)a(:,2) & b(2,:)a(:,2)\\ \end{pmatrix}$$

Therefore, to accomplish this "normal" matrix multiplication between a and b, in Yorick we would write b(,+) * a(+,) to do the "normal" matrix multiplication AB.

As we've noted, all these combinations above are only possible because the matrices are square. Remember that the indices marked for inner product must share the same length. If, for example a has 3 rows and 2 columns and b had 8 rows and 2 columns, the only possible operation would be a(+,)*b(,+).

An Example With SVsolve

The function SVsolve can be used to solve sets of simultaneous linear equations. Let's say that we have two arrays. Array a will represent the coefficients of the unknowns in our set of equations. Lets say that that we have n unknowns and m equations. The array a is an n x m matrix. Our linear equations, as we would normally write them (i.e., as I was taught in school) therefore look like this:

So if I were to pass the coefficient matrix (2D array) and the result vector (1D array) to SVsolve everything will work fine right? Err... no! Doh! Why not? It comes back to the issue above... the matrix multiplication is (at least for me) slightly unintuitive (probably just me!).

SVsolve() solves for A(,+)*x(+) = B. As we've seen, A(,+) will multiply the columns of A with x(+). Oops: SVsolve() is trying to do this:

So, the solution is to transpose() the array you are using before passing it to SVsolve()!

Rubber Indices In Yorick

Yorick has one other indexing syntax which has proven useful, which I call rubber indices. They address the problem of writing interpreted code which extracts slices of arrays when you don't know beforehand how many dimensions the array has. An example is an opacity array for which you know that the most slowly varying index represents photon wavelength, but there might be zero, one, two, or three faster varying dimensions representing position in space. -- PYTHON MATRIX-SIG

> a = [ [    [1,2,3], 
             [4,5,6]
        ],[
             [11, 22, 33], 
             [44,55,66]
      ] ]
> a
[[[1,2,3],[4,5,6]],[[11,22,33],[44,55,66]]]
> a(1)
1
> a(1,1,1)  ## Give me the first element in the inner array, from 1st array 
1           ## in the middle array, from the first array (of arrays) from the 
            ## outer array

> a(1,:)    ## Give me the first element in each inner array, contained in the
[1,4]       ## same parent array. This is a little incorrect because we don’t
            ## specify all array dimensions in the indices

> a(1,:,:)      ## Give me the first element in each inner array, from each 
[[1,4],[11,44]] ## outer array, contained in the overall array

> a(1,..)       ## The RUBBER INDEX says the same thing: Give me the first
[[1,4],[11,44]] ## element from the innermost array and do the same recursing
                ## outwards

> a(1,*)        ## Another RUBBER INDEX that collapses all arrays into one.
[1,4,11,44]

> a(:,1,1)      ## Give me everything from the first array, i.e., the array
[1,2,3]         ## itself, from the element in the middle array from the first
                ## element in the outer array.

> a(:,1,..)           ## Give me everything from the first array, ie. the
[[1,2,3],[11,22,33]]  ## array itself, from the first element in the middle
                      ## array from all elements in the outermost array.

Copying Oxy Objects

When you assigned an OXY object from one variable to another is is copied by reference. Thus changing an object member will be reflected in both variables as assignment is not copy for OXY objects.

> a = save(q = 123)
> a.q
123
> b = a
> save, a, q=321 /*<--  Changing a's member variable will change b's! */
> a.q 
321
> b.q 
321              /*<-- LOOK, b has been affected by the change to a! */

Given the rational behind Yorick as a number cruncher, this makes a lot of sense. It's a little like passing an array into a function. If we think as an OXY object as a potential "bag" of large data, copying it would be hugely expensive, so that is why an object assignment is NOT a copy!

To actually make a real copy of the object you have to manually copy each member of one object to the other.

> a = save(q=101)
> b = save()     /*<-- Must manually re-create a new... */
> save, b, q=a.q /*    ...object if we are to copy a */
> a
object [1]: group
> a.q
101
> b.q
101
> save, a, q=999
> a.q
999
> b.q
101              /*<-- Changing a.q has not affected b.q. We did a proper copy!

BUT, this is further complicated by arrays as member variables and OXY objects as member variables. For example, examine the following.

> a = save(q=[1,2,3])        
> b = save()
> save, b, q=a.q
> b.q
[1,2,3]
> a.q
[1,2,3]
> a.q(1) = 111
> a.q
[111,2,3]
> b.q
[111,2,3]

In the above example we used the same recipe as above, but because q is now an array the reference to the array is copied and not the value. Again, make sense for Yorick as a number cruncher of large data arrays, but if you wanted a copy, this could throw you. The same copy-of-reference problem would occur if q has been another OXY object.

You make have done a double-take here because previously, when describing arrays, we said array assignment copies the array values, i.e. a complete copy of the array is made. This is clearly not the case when saving an OXY object member as an array. OXY object members appear always to be references unless the member stores a non-array primitive.

If you want to copy an array "into" an OXY object (actually create a nameless array and assign the reference to the OXY object member), use array slicing and take, if you want the entire array, a slice of the entire array:

> a = [1,2,3]
> b = save(a=a(:)) # By taking a slice of the entire array we force a copy.
> b.a              # Slicing operations happen immediately and result in a 
[1,2,3]            # temporary array, the reference to which is then stored
> a                # in the OXY object member.
[1,2,3]
> a(1)=99
> a
[99,2,3]
> b.a
[1,2,3]

To fully, deep copy, an OXY object your therefore have to build a new object from scratch, and copy in all primitive types and then recursively copy in all member OXY objects and also take care to copy the arrays correctly.

If you are writing a lot of Yorick scripts that get included in other scripts etc etc, you can end up with a lot of variable and function names in the global scope... this can lead to global namespace pollution.

You can create a kind-of namespace in Yorick. There is no "namespace" command but you do have access to save() and restore() which can be used to save a set of variables in an object and restore those variables from that object respectively. The general idea is this:

/* a and blah are already in the global namespace */
a = 1
func blah(void) { ... }

/* Our script starts here...
 * Now we defined our own namespace that will be accessed through the
 * object my_new_namespace. The save() function saves the variable a and
 * function blah in this object. */
my_new_namespace = save(a, blah)
//{
   /* Here we overwrite the variable a and function blah in the global namespace
    * with these new values */
   a = 101
   func blah(does_something_else) { ... }
//}

/* We restore() the original namespace that existed before we modified it by 
 * swapping the data from my_new_namespace.a and my_new_namespace.blah with
 * their the current values in the global namespace - the ones we just defined.
 * Thus the global namespace appears untouched by our script, and the variables
 * and functions in our script can be accessed using my_new_namespace.xxx.
 */
my_new_namespace = restore(my_new_namespace)

The are a couple of subtleties to watch out for here and they have to do with when statements are evaluated. For example, if you assign from one variable in your module's "namespace" to another variable in your module's "namespace", you would not use the namespace prefix my_new_namespace.xxx. For example:

//------------
// my_module.i
//------------
my_new_namespace = save(a, b, blah, blah_oops)
   a = 222
   b = a
   func blah(void) { print, my_new_namespace.a; }
   func blah_oops(void) { print, a; }
my_new_namespace = restore(my_new_namespace)

//------------
// main_program.i
//------------
a = 111
b = 999
func blah(void) { print, a; }

require, "my_module.i"

print, "global a", a
print, "global blah"
blah, []

print, "my_module a", my_new_namespace.a
print, "my_module blah"
my_new_namespace, blah, []
print, "my_module blah_oops"
my_new_namespace, blah_oops, []

Here I cover "manually" writing a plugin, i.e, not using Codger to auto-generate the C code using the PROTOTYPE Yorick comment method. Most of the stuff you need to know in order to do this can be found in yapi.h and the source code comments are pretty comprehensive.

One point to note about terminology. Whenever I talk about "stack" I will, unless made explicitly clear, be talking about the Yorick stack. This is a stack maintained by the Yorick interpreter and has nothing to do with your process/thread stack!

An Intro: Create The Plugin Shell And The Basics

/*
 * FILE: jehtech.i
 */
plug_in, "jehtech"

extern jehtech_Version;
/* DOCUMENT void jehtech_Version()
 *
 *    Return string describing version of the library
 */

/*
 * FILE: jehtech.c
 */
#include "yapi.h"
#include "ydata.h"

#ifndef NULL
#define NULL '\0'
#endif

void Y_jehtech_Version(int argc)
{
   ystring_t *verStr = NULL; /* ystring_t is char*, so this is char** */

   /* Push a ystring_t* onto the stack. This is a char** */
   verStr = ypush_q(0);
   if( !verStr )
      y_error("Could not push return string onto stack"); 

   /* p_strcpy is Yorick's mem-managed of strcpy(). Returns char* */
   *verStr = (ystring_t)p_strcpy("v1.0");
}

$ yorick -batch make.i
created Makefile
 automatically generated make macros and dependencies:
PKG_NAME=jehtech
PKG_I=jehtech.i
OBJS=jehtech.o
 edit Makefile by hand to provide PKG_DEPLIBS or other changes

$ rlwrap yorick
 Copyright (c) 2005.  The Regents of the University of California.
 All rights reserved.  Yorick 2.2.01 ready.  For help type 'help'
> #include "jehtech.i"
> jehtech_Version()
"v1.0"

Passing Values To C Functions From Yorick

void Y_functionName(int argc) {
   /* argc gives the number of elements pushed onto the **Yorick** stack. 
    * Note this has nothing to do with the C stack! */
   ... 
   /* Anything left or placed on the top of the **Yorick** stack is the
    * function's return value */
}

/*
 * FILE: jehtech.i
 */
... snip ...
extern jehtech_ParameterOrder;

/*
 * FILE: jehtech.c
 */
... snip ...
void Y_jehtech_ParameterOrder(int argc)
{
   long hundreds = ygets_l(0),
        tens     = ygets_l(1),
        units    = ygets_l(2);

   ypush_long(hundreds*100 + tens*10 + units);
}

$ rlwrap yorick
 Copyright (c) 2005.  The Regents of the University of California.
 All rights reserved.  Yorick 2.2.01 ready.  For help type 'help'
> #include "jehtech.i"
> jehtech_ParameterOrder(1,2,3)
321

/*
 * FILE: jehtech.i
 */
...
extern jehtech_ArrayTest;

/*
 * FILE: jehtech.c
 */
...
#include <stdio.h>
...
void Y_jehtech_ArrayTest(int argc)
{
   long i;
   long dimInf[Y_DIMSIZE];
   long ntot;
   long *ary = ygeta_l(0, &ntot, dimInf);
   printf("jehtech_2DArray: ntot == %ld\n", ntot);
   printf("jehtech_2DArray: dimInf = [");
   for(i = 0; i <= dimInf[0]; ++i) {
      printf("%ld%s", dimInf[i], i == dimInf[0] ? "]" : ", ");
   }
   printf("\n");
}

$ make all
...
$ rlwrap yorick
 Copyright (c) 2005.  The Regents of the University of California.
 All rights reserved.  Yorick 2.2.01 ready.  For help type 'help'
> #include "jehtech.i"
> jehtech_ArrayTest, [1,2,3]
jehtech_2DArray: ntot == 3
jehtech_2DArray: dimInf = [1, 3]
> jehtech_ArrayTest, [[1,2,3], [1,2,3]]
jehtech_2DArray: ntot == 6
jehtech_2DArray: dimInf = [2, 3, 2]
> jehtech_ArrayTest, [[[1,2,3,4], [1,2,3,4]], [[1,2,3,4], [1,2,3,4]]]
jehtech_2DArray: ntot == 16
jehtech_2DArray: dimInf = [3, 4, 2, 2]

/* 
 * FILE: jehtech.i
 */
...
extern jehtech_Dump2D;

/*
 * FILE: jehtech.c
 */
...
void Y_jehtech_Dump2D(int argc)
{
   /* PRE: array is 2D! */
   long row, col;
   long dimInf[Y_DIMSIZE];                       /* [rank, #cols, #rows] */
   long *ary = ygeta_l(0, NULL, dimInf);
   for(row = 0; row < dimInf[2]; ++row) {        /* dimInf[2] is #rows   */
      for(col = 0; col < dimInf[1]; ++col) {     /* dimInf[1] is #cols   */
         printf("%ld\t", ary[row*dimInf[1] + col]);
      }
      printf("\n");
   }
}

$ rlwrap yorick
 Copyright (c) 2005.  The Regents of the University of California.
 All rights reserved.  Yorick 2.2.01 ready.  For help type 'help'
> require, "jehtech.i"
> jehtech_Dump2D, [[1,2,3,4], [5,6,7,8]]
1	2	3	4	
5	6	7	8

Cool... this looks promising as to test some Yorick I have been using Robot Framework so being able to do this would make tests easier! Haven't done anything with it yet but Munro has written a Python module interact with Yorick.