Untitled Document

Suppose that you have a Dataset SNP that contains a six column Dataset of SNP futures prices representing respectively, Open, High, Low, Close, Volume and Open Interest for a 40 month period. If we type

Then the result is a Dataset that contains all of the dates of the SNP Dataset, and the column labeled CLOSE or CL. If you wish, you may type CL instead of CLOSE (they are aliases). If column labels do not exist, FDC will choose the fourth column of the Dataset (or column 3 if there are only three), or give an error message. Similarly the functions OPEN, HIGH, LOW, VOLUME, and OI (for open interest) select the correspondingly labeled columns or certain defaults, if they exist, otherwise they return an error.

Several other features of a Dataset can be selected and displayed, and they are summarized as follows:

DATE forms a new Dataset, dated as before, but also containing the dates themselves as a single data column. Although this function seems too simple to be of much value, it can be tremendously useful. It allows you to process dates as part of a pattern selection process, or to select parts of another Dataset. It returns the dates in the form YYYYMMDD.

DAY, MTH, YR, and DW act in the same way as DATE, but return, as respective data for each date, the day of the month, the month, the year, or the day of the week associated with that date. In the case of an entry of 20021128, DAY will display "28". YR in this instance would display "2002".

will get you the 4th column of the corresponding IBM stock data. You may request more than one column, for example:

will display both the first and fourth columns. The numbers do not have to be progressive--you could request IBM COLS 4 1, which will return the columns in the order requested. Also you can request something like

and get columns 2 and 3, and two copies of column 4. Many Datasets will have the columns labeled. For example, the fourth column of the IBM Dataset might be labeled “VOLUME”. You could then request the selection of this column with the command

Another way of placing data together is WITH, which combines columns of Datasets. The columns of the Dataset on the right are inserted after the columns of the Dataset on the left, with common dates aligned. If either Dataset contains entries for dates that the other does not have, then only its columns will be visible on that date. That way, you can combine successive Datasets into one, and design your own way of handling the overlapping dates.

Suppose that you have a Dataset GOLD containing prices for gold futures. The command

is identical to GOLD COLS 1 4 (if the open and close are in the first and fourth column position). The command WITH could also be used with two different Datasets, as in:

An alias of WITH is the comma (" , " ). As in the above examples, you could have typed:

COMMON functions like WITH, but instead of combining all columns of the left Dataset with all columns of the right Dataset, COMMON combines the Datasets only on dates that both sets have in common. Syntax is [data A] COMMON [data B].

The output of this function has the dates in the first column and the rest of the Dataset in subsequent columns. This makes it convenient to copy or cut the data and paste it into a spreadsheet or word processing program. The output of this operation is identical to

When selected from the edit menu of a dataset, this function replaces the dataset with the results of the package function, and changes it to a temporary dataset.

ROW SELECTION AND MANIPULATION

Just as we have selected data by columns, we can select data by rows. The most logical first example of this is selection by dates, using the functions FROM and TO. A date normally consists of a number representing respectively the year, month and day being specified. Thus,

will choose from the stock Dataset MSFT just that part consisting of the lines whose dates lie between July 1, and December 8, in 1998. It does not matter if trading actually took place on those two dates. Thus, if the dates of MSFT range from 19980612 to 19981204 and 19980701 is not a date of a price of MSFT, the result of the above command will be all lines with dates greater than 19980701 and less than or equal to 19981204.

Should you just use FROM as in MSFT FROM 980103, then you will get all data starting with that date. Should you just request MSFT TO 980103, you will get all data from the beginning of the data series to the specified date.

If dates requested are not in the Dataset, FROM TO will produce no result. These functions will also preserve labels if they exist.

Suppose you only wish to look at the most recent data in a data series, or the earliest data. FDC provides the functions FIRST and LAST for this purpose. For example,

returns the entries from the 28th to the 50th. Here again, labels are preserved if they exist.

The DROPFIRST and DROPLAST functions respectively drop the first (last) specified numbers of items of the dataset.

You may need to eliminate certain rows from a data series, and this is done by using the function REMOVE. Any Dataset can be used before REMOVE and any list of dates used after. Then the listed dates and all data that corresponds to them will be removed from the data series, if they are contained in that series. Thus,

will remove rows corresponding to the three given dates from the Dataset SFRANC, if they occur in this Dataset. As a convenience, REMOVE will also function by using a second Dataset in the form A REMOVE B. The result is that all rows are removed from A that begin with dates that occur in B. Thus,

will remove from SEPTOIL any rows that correspond to dates that it shares with OCTOIL. Such operations are sometimes useful when splicing data series together.

The function MERGE "glues together" time series vertically while taking account of their dates. More precisely

forms a new series which starts with A (in its entirety) and adds data from B corresponding to any dates not contained in A. If, for example, the dates in A range from 20000201 to 20010430, and those in B range from 20010101 to 20010630, then A MERGE B will contain all of A followed by that part of B lying between 20010430 (not including this date if it is in A) and 20010630. Note that if there is a conflict, (i.e., if A and B have an entry that has the same date) then the first mentioned Dataset prevails. This is easily remembered by thinking that the first name mentioned in the merger is the dominant one.

With the MERGE verb we can put (say) September Silver together with December Silver, the latter beginning exactly when the former stops, without being concerned about (or even knowing) when the September series ends and the December series begins, or how they overlap.

MERGE also preserves the plottype (the charting format you may have previously selected for the data) of the first Dataset and its column headings where possible (the two merged Datasets may have different numbers of columns. Then the columns corresponding to those of the first Dataset are labeled accordingly.

MERGEIN functions like MERGE except that it lets the first-named dataset dominate with regards to data, but lets the second-named dataset dominate with regards to plottype and column labels. Thus, MERGEIN can be quite useful as a data series editor. Simply create a data series B which corrects some erroneous entries of A, and perhaps also includes some missing entries, and then enter B MERGEIN A. The new dataset will contain the corrections and updates, but it retains the main (profile) features of Dataset A.

SAVING DATA

At this point we need to know how data is handled when it is created or manipulated. When you modify a Dataset, the modifications will not be made permanent unless you save the result using the Menu or Save icon on the Dataset window. Any further use of the name of that Dataset will refer to the version in permanent memory. Thus you should save a Dataset or use “Save As” to save a changed version under another name, if you want to use the name multiple times in an input line.

When you create data as a result of executing an input line, the result is given a temporary name, “Tempn”, where n is a number chosen by the program. You can save these Datasets under actual names if desired. Whether you do or not, these temporary Datasets, as well as any permanent Datasets you are displaying, can be used in subsequent lines of input, as long as they are active.

THE ULTIMATE CALCULATOR

MATHEMATICAL FUNCTIONS

Suppose you have two Datasets named A and B, and you wish to add them together, type:

This function adds each dated line of the Dataset A to the line with the same date in Dataset B if it exists. If the A line has no matching dated line in B, then it is ignored. The same behavior takes place if the right data row has no matching date on the left.

The actual addition of rows takes place in the following manner. The Datasets A and B must either have an equal number of columns or one of them must have a single column. If the number of columns is equal, then corresponding columns are added. If Dataset A (say) has only one column, then B can have any number and the single column of A is added to each column of B. A similar thing happens if B has only one column. In addition, either A or B (or both) can be a number rather than a Dataset. Thus, FDC can double as an ordinary calculator.

To remember the conditions on the columns of these Datasets, just use the rules:

(1)       One against many
(2)       Many against one
(3)       Equals against equals

will add 4 to each entry of each column of A. If A has 5 columns and B has 1, then the command

will add the single column of B to all columns of A. Finally , if both A and B have 4 columns, then the command

will add column 1 of A to column 1 of B, column 2 of A to column 2 of B, and so on. All of this occurs only on dates common to both Datasets.

Many useful computations can be made with these functions. For example, the command

will compute, for each day, the average SNP price of the day, i.e. the midpoint of the range.

The function MAXIMUM (or MAX) operates between two numbers by choosing the largest, and MINIMUM (or MIN) chooses the smallest. Then when applied to Datasets, the behavior of these two functions is just as explained above for PLUS.

The function MOD is designed to work on integer data. N1 MOD N2 gives the remainder after dividing N1 by N2. The function INTDIV has similar restrictions. It is the smallest whole number less than or equal to the usual quotient. Thus 7 MOD 3 is 1 while 7 INTDIV 3 is 2. These functions are useful in forming macros, as we will show later.

OVERALL ANALYSIS OF A DATA SERIES: THE “TOT” FUNCTIONS

These functions operate on each data column and produce one output for each column. As the output does not correspond to a date, no date will be displayed.

For example, TOTMAX will display the overall maximum value in each column. Suppose that the Dataset GOLD has 4 columns of data (OP HI LO CL). Then the command

would result in your getting the maximum of each of the 4 columns. That is, you would get the highest OP, the highest HI, the highest LO, and the highest CL. If there were more columns, then you would get the highest for each, such as the highest VOLUME, and highest OI. TOTMIN functions similarly, to produce the overall minimum value of each column.

TOTSUM adds up all the entries in each column and produces a sum for each, while TOTAVE (TOTMEAN) produces the column average for each column. The command

will produce the average of all the daily ranges of the BOND series. Similarly, the command

TOTSTDV produces the standard deviation of each column. TOTSTDV and TOTMEAN (a synonym for TOTAVE) enable the user to compute useful statistical measures of the data in the columns of any Dataset. The function TOTCOR computes the overall correlation of corresponding columns of a Dataset, Syntax: [DS] TOTCOR [DS].

To compute the column-by-column correlation of two data series A and B, just enter

To compute the column-by-column covariance of two data series A and B, just enter

CUMULATIVE ANALYSIS OF A DATA SERIES: THE CUM FUNCTIONS

As with the TOT series, these functions apply to a single Dataset, one column at a time. Unlike the TOT series, the result in this case is another Dataset of exactly the same size. For example, CUMSUM computes, for a given date, the cumulative sum of entries from the beginning of the data up to that date. It does this for each column of the input data. Thus if WHEAT is a Dataset with 4 columns and 20 lines of entries, then the command

will result in another 4 by 20 Dataset with each date displaying the cumulative sum for each column from the beginning of the Dataset up until that date.

will define for you 98% of the highest high for GOLD up to each date. This is identical to a 2% decline from the highest high.

To compute the column-by-column cumulative correlation of two data series A and B, just enter

To compute the column-by-column cumulative covariance of two data series A and B, just enter

MOVING VERTICAL PERSPECTIVE: THE MOV SERIES

Moving averages are quite familiar to market technicians. With FDC they can be accessed easily. The "MOV" functions have two places for input. The left input is the number of days over which the operation is to be taken, and the right input is the data series. The data series can have one or several columns, and the left input can have one or several numbers. For example, the command

asks for a 3-day moving average of BONDS, and will produce the 3-day averages for each data column in BONDS.

will display the 3, 5, and 10-day moving averages for the closing price of BONDS as a 3 column Dataset (for those dates for which all averages exist, i.e. from the 10th day on). Finally, if BONDS has exactly 4 columns (say OPEN, HIGH, LOW, and CLOSE), then the command

will produce a four column Dataset in which the first column is the 3-day moving average of the opening prices of BONDS, the second column is the 4- day moving average of the highs of BONDS, and so on. The only dates returned are those dates on which all of the moving averages can be calculated. With regard to numbers expressed in the left input and the number of columns in the data series in the right input, the following rules apply, as in the PLUS series of functions:

That is, (a) you can operate one moving length against each of the columns in a multi-columned data series or (b) you can operate many different moving lengths against a one-columned data series, or (c) you can operate a fixed number of moving lengths against a data series with exactly the same number of columns. The command

will produce an error message, because the left input has three lengths and the right input has two columns.

In addition to MOVAVE, whose alias is MOVMEAN, the other moving functions are MOVSUM, MOVMAX, MOVMIN and MOVSTDV. Each of these functions operates over a certain fixed length that moves along with the data. However, instead of taking the average of a certain number of past entries, these functions will take respectively the sum, maximum, minimum, or standard deviation (from the average).

The MOVMAX and MOVMIN functions are particularly useful to those traders/analysts looking for chart breakouts. Thus, the command

will give you the moving 4-day lowest Low of the TBILL Dataset. With these functions, it is easy to form statistical information of all types. A somewhat unusual capability is given by MOVSTDV. Thus, the command

will give you, for each date, the standard deviation of the previous 5 closing prices about their mean value. We have included this capability, since we believe that it is an interesting, and previously untapped, measure of volatility. Some people prefer using a "standard deviation" given by adding the absolute values of all differences of the dataset with its average. This is given in moving form by MOVSTDV1.

The MOVTREND function determines the regression, or best fit, line for the previous N days, then returns its value today. This is a powerful additional smoother, that has interesting low-lag properties. If the dataset has multiple columns, this function will deliver moving trends for each column. MOVTRENDALL when operated on a dataset with multiple columns will average those columns prior to taking the moving trend, giving you a 1-column output only.

Similarly, the function MOVSLOPE returns the slope of the moving regression lines for each column separately, while MOVSLOPEALL returns the moving slope of the average of all of the columns (i.e. 1-column).

The moving trend has a syntax that is a bit different from the standard moving operators, and thus requires some elaboration. When one performs a moving trend, FDC calculates the formula for that operation. Naturally, with the formula, one could easily extend the trend, and we have enabled the user to access that capability. The simplest case is when the left argument is a number or a number pair. The input

calculates the 9-day moving regression line (or trend) for each column of SNP and gives you today’s value of that trend On the other hand, the input

computes the same trend lines, but on each day gives the value two days ahead. Thus 9 MOVTREND SNP is the same as (9 0) MOVTREND SNP . If you want different actions on each column or several actions on one column, just put one number or pair in the left argument for each column. A set of examples provides the most clarity:

The first example (a) performs a 4-day movtrend on the open, a 5-day movtrend on the high, a 6-day movtrend on the low, and a 7-day movtrend on the close.

Example (c) calculates a 5-day movtrend, and then extends it 2 days into the future, for each column of the 2-column dataset (SNP COLS 2 3)

The last example (d) calculates a 5-day movtrend extended 2 days on the open, a 6-day movtrend (unextended) on the high, a 7-day movtrend extended 3 days on the low, and an 8-day movtrend (unextended) on the close.

Thus, FDC will observe the “one against many; many against one; equals against equals” rule, but the items on the left may be either numbers or pairs.

What does "Adaptive" Mean?

Essentially it means that a parameter of the function can vary with time. Let us give a real example:
Suppose you are performing an operation on a data series, such as taking a moving average of the data. Normally you would specify the time period over which you would be performing this operation, as for example a 10-day moving average or a 20-month moving median, etc. The time period (i.e. 10 days or 20 months) is your variable.

Now suppose that what you are doing demands performing this operation over different time periods under different circumstances. In other words, you need to vay that is the number of days over which you take the moving average. Most software operations are set up to deal with user-defined parameters that do not change. So what you need is software that can perform such operations with non-constant or adaptive input. In theory this should not be a big deal. But it is, because most software cannot deal with such a request.

Let's go back to the example of a 10-day moving average. Instead of specifying 10 MOVAVE BONDS you will replace the 10 with another dataset (say, [data2]), whose dates correspond to the Bond dataset, and whose entries are integers, specifying how many days to use in the moving average, on each date.. Operationally you would then request [data2] MOVAVE BONDS Thus, on each date, FDC would look to the value in [data2] MOVAVE BONDS to perform the appropriate moving average on Bonds.

STEP-WEIGHTED MOVING AVERAGES

Some traders and technicians like their moving averages weighted according to their preferences. In fact, there are probably as many different styles as there are traders. With FDC you can have whatever you want. For example, suppose you want to look at an average of the last 5 day's prices of the Dataset TNOTES, but you consider yesterday's price twice as important as each of the preceding 3 days, and today's price 3 times as important as those first 3 days. The command

will give you exactly what you want. That is, the first 3 days are each weighted as 1, with the 4th date weighted as 2, and the most recent (or 5th) date weighted as 3. In this case, FDC takes 1 times each of the first 3, 2 times the 4th, and 3 times the 5th, then divides the whole thing by the sum of the weights (or 8). If the weights are all positive, this last division simply replaces them by proportional weights that add to 1. Note that the command

would give you the normal 3-day moving average where each day is considered equally. Note also, that the weighting numbers do not have to be integers, nor do they have to be progressive. That is, you could enter

In WTAVE, realize that the numbers refer to the weighting of the days they represent. In the expression

there are 5 numbers mentioned, meaning that we are only averaging over 5 days. Each day is weighted according to the number value expressed for that day. In this regard you must remember that this function operates slightly different from other FDC functions. The weighted average is applied separately to each column of the Dataset. (The standard, "One vs. many; many vs. one; etc." does not apply to WTAVE).

Of course some of the weights may be negative, which results in a somewhat different computation. For example, -1 0 0 0 1 WTAVE [data] produces the change in the 4 day moving average of the data, something usually called an oscillator. All oscillators can be produced in this way.

There are other weighted averages that are mixtures of moving averages and oscillators. One of the favorite gurus uses the following with considerable success: -2.5 -1.5 -0.5 3 5.5 4.5 1 WTAVE MIDRANGE [data].

As with all of the moving functions, weighted averages can easily be made adaptive. To do so, just request N WTAVE [data], where N is a dataset itself. On each date, N contains a row of numbers to be used as weights on that date.

WTSUM - This function behaves similarly to WTAVE, except that WTSUM takes the sum without averaging. The syntax is: a1 a2 a3 WTSUM [data]. Naturally, making the left argument a dataset can make this function adaptive. As with WTAVE, WTSUM does not follow the standard rules for matrix operations, but applies the weights to each data item of each column. The weighted sum function is necessary for the calculation of several technical indicators including the Center of Gravity indicator.

EXPONENTIALLY-WEIGHTED MOVING AVERAGES

A lot of traders like to weight their data according to what they consider a more effective process, where the weighting decreases exponentially (going from recent to older data). This so-called exponential moving average (which is not a moving average at all) is produced by computing for each day, a multiple of that day's price plus another multiple of the previous exponential moving average, where the multiples add to one.

Specifically, N represents the percentage of the previous day's value to be computed into today's value. The request .8 EXPAVE BONDS will take 80% of the previous day's value and add to it 20% of today's value.

In FDC, the exponential moving average can be applied to a Dataset in exactly the same way as an ordinary moving average. Thus, the command

computes an exponential moving averages for the Dataset CL EUROS. As with ordinary moving averages, you can use one period against many columns, many periods against one column, and equals against equals.

As with the moving functions, the left argument can be a dataset, which will have the result of making the EXPAVE function adaptive. Just be mindful of the standard rules for matrix operations in this section

Many researchers refer to an "x-day exponential moving average". Such an animal does not exist, since exponential averages effectively consider all the data points available from the beginning of the data. However that does not solve your problem, which is to complete the recipe you have been given (e.g. "use a 12-day exponentially weighted moving average"). The good news is that FDC will also accomodate users accustomed to the "days" method. Instead of keying in a value for N between 0 and 1, just key in an integer greater than 1. FDC knows that ".154 EXPAVE BONDS" is equivalent to "12 EXPAVE BONDS". The table below illustrates the relationship between the two, which is loosely based on picking a value for N that results in approximately the same lag as the corresponding ordinary moving average has for the given number of days.

WEEKLY, MONTHLY, AND YEARLY OPERATIONS

WKAVE (alias WKMEAN) averages the values that occur during each week of a given Dataset and within each column of that Dataset. A week consists of all the dates that lie between a given Monday and the following Sunday and belong to the Dataset. Depending on the week, and the Dataset, the number of days of that week present in the data can vary from 0 to 7. FDC automatically handles these differences because the function WKAVE is a smart one and "knows" which dates are which days of the week.

The date assigned to the weekly average is the last date of that week for which data is present. WKSUM, WKMAX and WKMIN operate in exactly the same way.

These dates fall respectively on Monday, Tuesday, Wednesday, Friday, and then Monday through Friday. The result of executing the command

WKBARS takes a 4-column dataset of daily data and turns it into a 4-column dataset in which the open is the week’s open (the open on the first day), the high is the weekly maximum of the daily highs, the low is the weekly minimum of the daily lows, and the close is the last price of the week. Thus WKBARS is an easy way to turn daily data into weekly data. MTHBARS and YRBARS work similarly.

The monthly functions of MTHAVE (alias MTHMEAN), MTHSUM, MTHMAX and MTHMIN all work similarly, but they combine all dates that lie within the same month, and assign as the date a synthetic date of the 32nd of the month. This synthetic date is technically useful when monthly data is combined with daily data.

The yearly functions of YRAVE (alias YRMEAN), YRSUM, YRMAX and YRMIN also work similarly, but they combine all dates that lie within the same year, and assign as the date a synthetic date of the 13^th month, day 00.

VERTICAL (CHRONOLOGICAL) DATA COMPARISONS

The function BACK shifts data within columns down a certain number of days (or weeks, etc.), while holding the dates constant. Thus, the command

produces a Dataset that, for each date and for each column lists the GOLD prices that occurred 3 days earlier in that column. Of course, the first two dates of the GOLD Dataset will be omitted since the required data cannot be computed for them. FDC functions always compare Datasets at common dates, so this shifting allows comparison with earlier dates, something economists call a "lag". This is quite useful in a variety of situations. For example, the command

produces, for each date of the GOLD series (from the 4th one on) the difference between the current price and that of 4 entries earlier. The command

shows, for each relevant date the extent to which today's BONDS close differs from the highest high of the previous 4 days.

SILVER WITH (GOLD BACK 5) is the same as GOLD WITH (SILVER AHEAD 5), except the date displayed will always be that of the first Dataset mentioned.

N may also be a dataset, which means that the function can be adaptive; that is it can have a different integer on each date. Should N be a one-column dataset, BACK will move the data for each date back the number of periods specified by N for that date. Should N be a dataset of several columns, it will operate on the other dataset according to the standard rules for matrix operations in this section.

It is possible for N to be a negative integer. In such a case, [data] BACK -2 is equivalent to [data] AHEAD 2. Also, if N is not an integer, it will be replaced by the closest integer. These rules hold so that one can freely use the output of another function as N, a process that is often necessary when computing adaptive indicators.

The AHEAD function moves a Dataset ahead in time. The syntax is [data] AHEAD N (where N is a positive integer), and the result displays, on each date, the data from N dates in the future. Should you be studying whether one Dataset will either lead or lag another, you will need to first compare them. For example, economists consider the stock market a leading indicator that leads the economy by six months.

N may also be a dataset, which means that the function can be adaptive; that is it can have a different integer on each date. Should N be a one-column dataset, AHEAD will move the data for each date ahead the number of periods specified by N for that date. Should N be a dataset of several columns, it will operate on the other dataset according to the standard rules for matrix operations.

It is possible for N to be a negative integer. In such a case, [data] AHEAD -2 is equivalent to [data] BACK 2. Also, if N is not an integer, it will be replaced by the closest integer. These rules hold so that one can freely use the output of another function as N, a process that is often necessary when computing adaptive indicators.

CHANGE takes the algebraic difference between the value shown for each day and the value for the preceding day (for each column). In stock market terms, today's price minus yesterday's price. The syntax is simply CHANGE [data]. This function operates identically to [data] MINUS ([data] BACK 1).

This function may also take a left argument. For example, N CHANGE [data] will produce identical results to [data] MINUS ([data] BACK N). That is, 5 CHANGE [data] will subtract the entry from five periods ago from the current entry. Furthermore, the left argument N can also be a dataset, specifying a different value of N for each date. This facility makes the CHANGE function adaptive.

If N is 0, the data is not changed, so the change is 0. If N is not a nonnegative integer, it will be replaced by the nearest such integer. These rules avoid errors when using the output of another computation as N.

The function UPRUN computes, for each date, the number of successive dates on which the data of each column has been "up", or greater than the previous date. Thus, if the output is a "0", then that particular column's data is less on this date than it was on the previous date. If the output is a "3", then this is the third straight up day after a down day. DOWNRUN works the same except for "down" days.

will result in the following, 3-column output, provided that CL GOLD is as shown in column 1:

19960801	450.0	0	0
19960802	451.3	1	0
19960803	451.5	2	0
19960804	453.8	3	0
19960805	452.9	0	1
19960808	452.1	0	2
19960809	453.7	1	0

will produce the maximum number of consecutive higher closes that occurred over the life of BONDS.

THE HORIZONTAL FUNCTIONS OR "H" SERIES

Instead of working vertically, these functions work horizontally, using data that corresponds to the different columns of the same date. The result of applying any of these functions is always a Dataset with exactly one column. For example, operating on a Dataset of Bonds having 4 columns:

HMAX and HMIN will give, respectively, the largest and smallest item in the columns corresponding to each date.

Note that the following two requests produce identical results if the high and low prices for Bonds are in the second and third column respectively:

There are often several equivalent ways to perform tasks in FDC; and with practice, you will be able to formulate the most efficient one. Suppose you want to take the average of the HIGH, LOW, and 2 times the CLOSE of the Dataset BONDS, for each date. This can easily be done with the command

HDIFF takes the difference horizontally of the columns in a 2-column Dataset (The first column minus the second.) Unlike the other functions of this series, using this function on data series of more than 2 columns will result in an error message. For 2-column Datasets, the command

COUNT is a little different, but is closer to the H-series than to any other type of function. It outputs a single column that contains the numbers 1, 2, 3, ... up to the last entry of the Dataset. This is useful for generating various technical indicators.

INDIVIDUAL POINTWISE FUNCTIONS

These following mathematical functions apply to every number in every column of a Dataset, and round out the "calculator"-like qualities of FDC

2) SQR likewise takes the square-roots of each entry. This function requires a Dataset with all nonnegative entries.

3) ABS takes the absolute value of each entry (i.e. the magnitude without the sign).

4) ROUND rounds each entry to the nearest whole number, resulting in a data series of integers.

5) TRUNC drops the decimal part of any number, leaving only the integer part.

6) POS gives back identically any positive numbers, but replaces any negative entries with 0. Think of it as saying, "give me positive values only."

7) NEG returns negative numbers unchanged, but changes positive entries to 0.

8) SIGN returns a 1 if the number is positive, a -1 if it is negative, and a 0 if the number is 0.

13), 14), 15) SIN, COS, TAN provide the appropriate trigonometric functions for each entry.

16), 17), 18) SINH, COSH, TANH provide the appropriate hyperbolic functions for each entry.

19), 20), 21) ARCSIN, ARCCOS, ARCTAN provide the appropriate inverse trigonometric functions for each entry.

22), 23), 24) ARCSINH, ARCCOSH, ARCTANH provide the appropriate inverse hyperbolic functions for each entry.

FINDING PATTERNS / ANSWERING QUESTIONS

We make a special point of "question" or "logical" functions in FDC. These are functions that for each date check a particular condition of the data, and return a "1" for TRUE if the condition holds and a "0" for FALSE if it does not. Finding patterns and answering questions are really the same operation. That is, a pattern can only exist when the conditions defining the pattern are true.

FDC has a wide variety of question functions with which you can formulate almost any condition and search for almost any pattern. The function LT (less than) takes the common dates of the data series, or compares each date of one Dataset with the last available date from the other Dataset, like most two-input FDC functions. On these dates, the Dataset returned by the input

has the value 1 if the statement is true, (that is if the data of A is less than the data of B for those particular dates and the columns in question) and 0 if the statement is false. Thus, suppose that A and B are as shown below:

(There is no output for the first two dates of A since there is no corresponding or previous date of B to use.)

A description is given below of the logical functions that behave as just described, and their aliases.

If you are more comfortable with words than symbols, then go with your preferences. Logical functions can be combined to describe, and therefore search for, patterns of almost any complexity.

The function AND combines two logical series, producing an output that is also a logical series. It returns a 1 only when both series are true. OR operates similarly, but produces a 1 if either input series is true. Finally NOT reverses the roles of 1 and 0. These functions let you combine conditions and express them naturally. These conjunctions, along with the previous functions, allow you to describe patterns of arbitrary complexity. For example, the command

will output a 1 only on those dates when the three-day moving average of the closing price of SNP is higher than it was on the previous day, and the day of the week is a Tuesday. Similarly, the command

Many times in a trading or research situation, the overriding consideration is the occurrence of some type of crossover, such as the crossover of various moving averages. That is, when the 5-day moving average crosses over the 20-day moving average (e.g., going from below to above), that may mean that something is happening, in addition to the simple fact that the market is now higher on average for the last 5 days than for the last 20 days. FDC has CROSSUP and CROSSDN to locate these happenings. Specifically, you will use the command

This will result in a logical series that is a 1 on any date when the entry for A (in a given column) is higher than the corresponding entry of B, but on the previous date, this was not true. CROSSDN works in a similar manner, and each function handles different numbers of columns in the same way we have previously described. The command

will give a TRUE each time that the 5-day moving average of WHEAT is higher than the 20-day moving average, and on the previous day this was not true - the "crossover days". Do not confuse this with the command

which gives a 1 when the 5-day average of the stock ABC is higher than the 20-day moving average, regardless of what happens on any other day. For convenience, the right hand input for CROSSUP and CROSSDN can be constants. Thus you can enter

The result will be true when the high of WHEAT is greater than 435 today, but was less yesterday. This feature can be conveniently used to locate the first days on which a price series crosses some computed quantity. The expression

will return a true only if the closing price of the SNP on a given day first crosses below the last available weekly closing average.

The function CONSEC applies to any logical series, and produces a numerical output. If the series to which it is applied is FALSE on a given day, then CONSEC returns 0. If it is TRUE, but was false on the previous day, then CONSEC returns a 1. If the result is true for the third straight day, CONSEC returns a 3, and so on. This is similar to the behavior of the functions UPRUN and DOWNRUN. In fact, the command

Now that we have seen how to produce logical series we will see how to use a logical series to select from any data series. The function WHEN takes any data series as left input, and a logical series as right input. It returns that portion of the left input that corresponds to days when the right input is TRUE (that is, a 1). In this case, WHEN selects only dates common to the two inputs, and deals with multiple columns in the usual way.

Example: Let A be a list of daily prices for a stock over some period of time. The command

selects just that part of A corresponding to days when the close was higher than the open (note the use of parentheses). In this example, the command would have run correctly without parentheses, but it is always advisable to use them to avoid confusion. Consider them an insurance policy which prevents any misinterpretation of your intention.

TRENDING AND RELATED FUNCTIONS

TREND fits a trendline to each column of a complete data series and then provides to the user the values of that trendline, with dates identical to those of the input series. A trend is fitted to the data by "least-squares regression" in which all possible trendlines are fitted to the data, and the line chosen as the best fit is the one with the least total error (by summing the squares of the individual errors) to the original data. The command

will display the difference between the closing GOLD prices, and the values of the trendline of the dataset CL GOLD .

Alternatively, you may enter: (CL GOLD) - (TRENDALL GOLD), which will average all columns of the dataset GOLD prior to calculating the trend. That is, if GOLD is a 4-column dataset, then the TRENDALL function will use the four data points for each day to help determine the trend. In summary, TRENDALL uses all the data it can get for each day to determine the trend.

The function CHANNEL operates much like TREND, producing a channel or band around the best-fit line. The output is a series with two columns whose data are respectively the values of the trend line plus N, and the values of the trend line minus N, where the left input N is the number of points difference between the channel lines and the trendline. CHANNEL will only work with a data series that has one column.

will give you a data series of 2 columns, 40 points apart. That is, you will get a data series of 20 points above and 20 points below the trend of JYEN. This is equivalent to (TREND (CL JYEN)) PLUSMINUS 20.

After using TREND, check the Note tab for the trend parameters. Trends and channels can also be forecasted or extended into the future by using the function EXTEND. When a Dataset in which each column is a straight line is saved, the lines can be extended to give a projection into the future. For example, if the Dataset produced by the command

You will receive the dated series of BONDTRND, with 10 more entries at the end. These entries will be "dated" by numbers that simply represent their position in the Dataset. That is, if the Dataset BONDTRND had 56 entries, then the extended data will have "dates" listed as "#1, #2, ... , #10." This is a consistent scheme that allows this extended data to be used with all other functions. Any Dataset whose entries are determined by a mathematical formula can be extended into the future using the EXTEND command (FDC automatically remembers the formula and applies it).

FDC has one very unusual property, which we describe in detail later. You can draw trend lines (or any lines) by hand on the chart of a Dataset. Then when you return to the data side, you will see that the drawn lines have been turned into actual (synthetic) data. By using the EXTEND function, you can then extend hand-drawn lines.

FDC also has the ability to fit trendlines of higher degrees. For example, should the data resemble a parabola, it obviously should not be fitted with a straight line (which would be a linear equation – or one having a degree of 1). More likely than not, it can be fitted with a quadratic equation (i.e. one having a degree of 2). The syntax for this operation is N POLYTREND [DS], in which N is the degree. In the case of the quadratic, request 2 POLYTREND [DS]. Should your data have two profound curves in it, you may have more success with a cubic fit (the degree N equals 3). Making N=1 will get you a linear fit, identical to the request TREND [DS].

Note that similar to TRENDALL, the POLYTRENDALL function uses all data of a given date to determine fit. The output dataset of TREND and POLYTREND will always be for each column in the input dataset. Should you wish to use one line to describe all of the columns of a dataset, request N POLYTRENDALL [DS].

Should you wish to weight the values of the error terms in the fit, you may do so by specifying alpha. Alpha (A) is the percentage you wish to give to each preceding date’s error term. That is, if alpha is given a value of .9, yesterday’s errors will be valued as only 90 percent of today’s errors. The next preceding error will be valued at 81 percent of the present (.9 times .9), and so on. Why? The logic for discounting past error terms is embodied in the argument that today’s data is more important than previous data. However, note that you don’t want to value alpha too low, or your fit of data farther back from the present may not be representative of the data. Most researchers suggest an alpha value between .9 and 1. Be cautious with this feature: it’s like habañero sauce - more than a little is too much. Note also that if you do not specify a particular alpha value (and indeed you do not have to specify an alpha value), then FDC assumes you have specified a value of 1. That is, the data series will be unweighted. In the scientific literature, this process is referred to as “weighted least-squares regression”. The syntax is A TREND [DS], or A N POLYTREND [DS], and similarly for TRENDALLand POLYTRENDALL.

In all cases of fitting data, remember to use your eyes and brain. To paraphrase one famous (or infamous) lawyer, “If it doesn’t fit, you have to quit.”

CYCLIC ANALYSIS

We all know that markets respond to “real” information. But much of that “real” information is cyclically available. Common examples are weekly money supply, 10-day auto sales, monthly balance of trade and housing reports, quarterly industrial production, etc. In addition, a lot has been made of some of the very long-term fluctuations described as Kondratieff and Kuznets cycles. What you believe in is your choice; but market prices can be mathematically analyzed for cyclicality, and FDC does this with the function CYCLERUN.

This function performs a Discrete Fourier analysis of data using the FFT algorithm, and chooses the most dominant N cycles, where you specify N. It then takes the best possible fit of the data for those particular cycle lengths (using a regression or “best fit” technique) and creates a Dataset with them. This Dataset is the "best cyclic approximation to the data" that can be constructed using N cycles.

The constructed approximation is representative of the data, and can be quite useful for simulation and (if you are particularly careful) prediction. Of greatest interest is the fact that the output of the function is a smoothed representation of the raw data, without any lags. (Most data smoothing operations have lags.) To analyze closing bond prices for the most dominant 3 cycles, simply enter the command

The left-hand input is the number of cycles you wish to study. The right-hand input is the Dataset you wish analyzed. The left-hand input should be an integer.

Although the main purpose of this function is simulation and prediction, part of the output of CYCLERUN is the listing of the cyclic parameters. These can be found by selecting the Note tab, which will identify the cycles by cycle length (periodicity) and percentage of importance. For example, if the Note identifies a 17-day cycle as having contributed 36%, that means that 36% of the price action can be explained by that 17-day cycle. You may then wish to create a filter (e.g., a moving average or an oscillator) to screen future data for cyclic behavior of that length. N.B., a cycle of 17 days can be filtered out with a 17-day moving average.

Note that prior to performing the Fourier analysis, CYCLERUN first detrends the raw data. This is deliberate since it is presumed that you do not want trends confused with cyclic behavior. In the subsequent construction of the output data, CYCLERUN then adds back the previously subtracted trend. Should you wish to see the effects of cyclic analysis exclusive of trends, you should detrend first before performing the function. Should you wish to do so, a command like the following will work:

You will find it instructive to plot the cyclic approximation along with the original series. That way, you can visually inspect how closely the data can be explained by purely cyclic behavior. When using CYCLERUN, be realistic and reasonable. "A little goes a long way". That is, it might be reasonable to assume that five or eight cycles influence the data. It is not reasonable that the one-hundredth cycle has any relevance. Also, recognize that a dominant trend will always “trump” cyclic behavior, and avoid placing too much reliance on cyclic behavior in such circumstances.

Caveat: The CYCLERUN function uses the Fourier transform, which assumes all cyclic behavior is repetitive from the beginning of time to the end of time. That’s great for fitting data, but not generally reliable for forecasting markets. you see, market and economic data tends to waver between trending and cycles. The very reliable Kuznets cycles (of the U.S. economy) were a result of waves of immigration. Those waves would have been self-perpetuating, except that the U.S. changed its immigration laws. We haven’t seen Kuznets behavior since. However if you know the market is behaving cyclically, then cyclic forecasting can be profitable. A considerable amount has been written about cyclic analysis, but it is beyond the scope of this software manual. Please feel free to contact us for several references.

Suppose the Oracle at Delphi tells you that a cycle of a particular length exists, and you want to see how well the Oracle knows his stuff. That is, the prognostication is that a 21-day cycle for gold exists. You then need to fit the best possible 21-day cycle to your data. Simply request 21 CYCLE GOLD. Any particular cycle viewed individually will most likely not be that revealing. But reconstructing a dataset from various cycles and trends can be surprisingly informative.

N	# of	N	# of	N	# of
Value	days	Value	days	Value	days

0.667	2	0.091	21	0.049	40
0.500	3	0.087	22	0.048	41
0.400	4	0.083	23	0.047	42
0.333	5	0.080	24	0.045	43
0.286	6	0.077	25	0.044	44
0.250	7	0.074	26	0.043	45
0.222	8	0.071	27	0.043	46
0.200	9	0.069	28	0.042	47
0.182	10	0.067	29	0.041	48
0.167	11	0.065	30	0.040	49
0.154	12	0.063	31	0.039	50
0.143	13	0.061	32	0.038	51
0.133	14	0.059	33	0.038	52
0.125	15	0.057	34	0.037	53
0.118	16	0.056	35	0.036	54
0.111	17	0.054	36	0.036	55
0.105	18	0.053	37	0.035	56
0.100	19	0.051	38	0.034	57
0.095	20	0.050	39	0.034	58

19980201	20
19980202	22
19980203	18
19980205	9
19980208	34
19980209	11
19980210	8
19980211	24
19980212	30

19980205	69
19980212	107

Function	Alias

LT (less than) GT (greater than) LE (less than or equal to GE (greater than or equal to) EQ (equals) NE (does not equal)	< > <= >= = <>

ORCHESTRATING DATA - THE MANIPULATION OF DATASETS

COLUMN SELECTION AND MANIPULATION