Following the TDA User’s Manual
In memoriam Götz Rohwer
„So, wir fangen jetzt an.“
TDA, and this package
TDA – Transition Data Analysis – is Götz Rohwer’s program for the statistical analysis of event histories: transition rate models, parametric and semi-parametric, with episode splitting, competing risks, multi-episode data and time-dependent covariates, and around them the rest of a working statistician’s toolkit – descriptive statistics, life tables and product-limit estimation, regression of every ordinary kind, sequence analysis, loglinear models, graphs, spatial data, and its own PostScript plotting. It is a command-file program: a text file of commands, a data file, and a listing of what it computed.
Its history, as the manual’s preface and TDA’s
history.txt give it: begun in 1989 with early support from
the Hamburger Institut für Sozialforschung; grown inside Hans-Peter
Blossfeld’s project Household Dynamics and Social Inequality at
the European University Institute in Florence, with European Commission
support, and then at the University of Bremen. Version 5.7 (June 1994)
was the first the authors called mature, and it is the program behind
Blossfeld and Rohwer’s Techniques of Event History Modeling
(1995; 2nd ed. 2002). The 6.x line was a redesign, and from version 6.2
the program and the manual have two authors, Rohwer and Ulrich Pötter.
The 6.x releases, 6.1 in 1997 to 6.4p in March 2009, fall in Rohwer’s
years as professor at Ruhr-Universität Bochum (1997–2013), where TDA’s
homepage still is. TDA has always been a non-commercial project under
the GNU General Public License; its source is C, and the User’s
Manual of some 800 pages is its documentation.
tdaR is that program, compiled into an R package. The C is TDA’s,
made re-entrant so that it can run inside R and be called more than
once; nothing in it has been rewritten, and where it has been changed
the change is a repair of a demonstrated fault, listed in the package’s
doc/changes-from-tda.md. Each R function writes the command
file TDA would have been given, runs it, and returns what TDA produced –
the printed listing, the numbers behind it, and the tables – as an R
object with the usual methods. The results are TDA’s, at TDA’s
precision; the vignette Beyond the Manual shows how that is
checked.
This vignette follows the TDA User’s Manual of Götz Rohwer
and Ulrich Pötter and shows the same examples written against this
package. The section numbers are the manual’s own, so that a page of the
manual and the corresponding part of this document can be read side by
side, and the data are the manual’s files, shipped with the package
under extdata/exam. Boxes hold code and output, as they do
there.
Chapter 3 is the data structures, episode, sequence and graph data; chapter 4 TDA’s PostScript plotting; chapter 5 the matrix language and its expression, smoothing and minimisation commands; chapter 6, statistical procedures, is the bulk of the document; chapter 7 is relational data, graphs and clustering, and chapter 8, set-valued data, closes it.
3 Data Structures
3.3.3 An Example Data Set
The manual’s main example data set is taken from the German Life History Study (GLHS), collected at the Max-Planck-Institut für Bildungsforschung in Berlin in 1981–1983 (Mayer and Brückner 1989); the manual thanks Karl Ulrich Mayer and Hans-Peter Blossfeld for providing it, and it is the data set used in Blossfeld and Rohwer, Techniques of Event History Modeling. It holds 600 job episodes from 201 respondents. Each record is one employment episode, and the consecutive jobs of a respondent’s career follow each other in the file.
tda_rrdat() reads rrdat.1, names its twelve
columns, and derives the variables the manual’s nvar block
derives.
Box 1The data file rrdat.1, read and named
d <- tda_rrdat()
head(d[, 1:12], 9) ID NOJ TStart TFin SEX TI TB TE TMAR PRES PRESN EDU
1 1 1 555 982 1 982 351 555 679 34 -1 17
2 2 1 593 638 2 982 357 593 762 22 46 10
3 2 2 639 672 2 982 357 593 762 46 46 10
4 2 3 673 892 2 982 357 593 762 46 -1 10
5 3 1 688 699 2 982 473 688 870 41 41 11
6 3 2 700 729 2 982 473 688 870 41 44 11
7 3 3 730 741 2 982 473 688 870 44 44 11
8 3 4 742 816 2 982 473 688 870 44 44 11
9 3 5 817 828 2 982 473 688 870 44 -1 11All dates are century months: 1 is January 1900, 13 is January 1901,
and so on. TStart and TFin are the first and
last month in the job, TI the interview date,
TB the date of birth, TE the date of entry
into the labour market, and TMAR the date of marriage, zero
for the still unmarried. PRES is the prestige score of the
current job and PRESN that of the next one, -1
when there is no next job.
Three derived variables carry the episode: TFP is the
duration, TFin - TStart + 1; DES is the
destination state, 1 for a job change and 0 for an episode still running
at the interview; LFX and PNOJ are labour
force experience and the number of previous jobs.
Example 1 Where TDA needs an edef
command to turn columns into episode data, the episode is named in R by
the left hand side of a formula: Surv(TFP, DES) is
ts = 0, tf = TFP, org = 0, des = DES. Every command that
takes episode data reports what it was given.
Box 2Episode data, single destination state
tda_rate(Surv(TFP, DES) ~ 1, d, model = 2)$episodes sn org des episodes weighted mean.duration ts.min tf.max excluded
1 1 0 0 142 142 128.17606 0 428 FALSE
2 1 0 1 458 458 49.30349 0 350 FALSE458 episodes end in an event, a transition from 0 to 1, with a mean duration of 49 months; 142 are right censored, and their mean of 128 months is no estimate of anything but still says something about the data.
Example 2 The manual’s ed2.cf changes
nothing but the destination state, splitting the job changes by the
prestige of the next job. tda_rrdat(states = 4) is that
data set.
Box 3Episode data, alternative destination states
d4 <- tda_rrdat(states = 4)
tda_rate(Surv(TFP, DES) ~ 1, d4, model = 2)$episodes sn org des episodes weighted mean.duration ts.min tf.max excluded
1 1 0 0 142 142 128.17606 0 428 FALSE
2 1 0 1 84 84 45.04762 0 326 FALSE
3 1 0 2 219 219 45.91324 0 350 FALSE
4 1 0 3 155 155 56.40000 0 332 FALSE84 episodes end in an upward move, 155 in a downward move, and 219 do not change the prestige level. The number of censored episodes has not changed.
ed3.cf defines the same data as multi-episode instead:
one episode per job rather than one per person, told apart by the spell
number. TDA prints a table per spell, each closed by its own total (the
manual’s Box 8).
Box 4ed3.cf: multi-episode data
eb <- suppressWarnings(
tda_rate(Surv(TFP, DES) ~ 1, d, id = "ID", spell = "NOJ"))
eb$episodes sn org des episodes weighted mean.duration ts.min tf.max excluded
1 1 0 0 16 16 233.43750 0 428 FALSE
2 1 0 1 185 185 52.72973 0 326 FALSE
3 2 0 0 36 36 150.16667 0 407 FALSE
4 2 0 1 126 126 52.30159 0 350 FALSE
5 3 0 0 38 38 121.73684 0 329 FALSE
6 3 0 1 69 69 50.36232 0 332 FALSE
7 4 0 0 24 24 85.33334 0 340 FALSE
8 4 0 1 38 38 37.71053 0 146 FALSE
9 5 0 0 9 9 110.22222 0 328 FALSE
10 5 0 1 23 23 22.60870 0 70 FALSE
11 6 0 0 8 8 45.75000 0 127 FALSE
12 6 0 1 12 12 43.75000 0 112 FALSE
13 7 0 0 7 7 114.14286 0 295 FALSE
14 7 0 1 4 4 52.75000 0 96 FALSE
15 8 0 0 3 3 65.00000 0 172 FALSE
16 8 0 1 1 1 72.00000 0 72 FALSE
17 9 0 0 1 1 34.00000 0 34 TRUEattr(eb$episodes, "sums") # the totals under each spell; sn = NA is all spells sn episodes weighted
1 1 201 201
2 2 162 162
3 3 107 107
4 4 62 62
5 5 32 32
6 6 20 20
7 7 11 11
8 8 4 4
9 9 1 1
10 NA 600 6003.6.2 Graph Data
A graph is given to TDA as an edge list, gdd = I, J, V:
two columns of node numbers and one value per edge, a negative value
standing for no edge at all, which is how an isolated node is declared.
The manual’s Example 1 has five nodes and four edges, node 9 the
isolated one, and its Figure 1 is gd2.cf’s drawing of it as
a directed, valued graph.
Box 5gd2.cf
p <- tda_ps(xlim = c(0, 6), ylim = c(0, 4), width = 60, height = 30)
nodes <- data.frame(id = c(1, 7, 3, 4, 9), x = c(1, 3, 5, 2, 4),
y = c(3, 3, 3, 1, 1), grey = 0.8)
edges <- data.frame(from = c(1, 3, 1, 4), to = c(7, 7, 4, 7),
label = c(10, 0, 5, 13), arrow = "1.5,1.0")
p <- tda_pl_graph(p, nodes, edges)
plot(p)
3.6.5 Drawing Graphs
TDA draws a graph with plg, nodes placed by hand and
edges with optional curvature, arrowheads and values;
tda_pl_graph() takes the nodes and the edges as data
frames. This is the manual’s Figure 3.6.2-2, the graph of
gd1.dat drawn by gd3.cf: dashed edges carry
the second value variable.
Box 6plg: the graph of gd1.dat (gd3.cf)
p <- tda_ps(xlim = c(0, 9), ylim = c(0, 4), width = 80, height = 40)
nodes <- data.frame(id = c(1, 7, 8, 5, 11, 12, 9),
x = c(3, 6, 8, 1, 3, 6, 8),
y = c(3, 3, 3, 1, 1, 1, 1), grey = 0.8)
edges <- data.frame(from = c(1, 1, 7, 8, 11, 5, 7, 8, 12),
to = c(5, 7, 1, 8, 12, 1, 1, 7, 11),
curve = c(3, 0, 3, 0, 0, 0, -3, 0, 3),
label = c(3, 5, 6, 2, 13, 4, 15, 0, 14),
lty = c(1, 1, 1, 1, 1, 5, 5, 5, 5),
arrow = "1.5,1.0")
plot(tda_pl_graph(p, nodes, edges))
3.6.6 Trees
The same tree asgd15.cf, laid out radially:
pl = 1 rather than gd15’s pl = 3.
Box 7gd15a.cf
edges <- data.frame(
from = c(6, 6, 7, 7, 8, 8, 9, 9, 1, 1, 1, 7),
to = c(2, 4, 3, 5, 1, 6, 7, 8, 10, 11, 12, 13),
v = c(1.0, 1.5, 2.0, 2.5, 1.5, 1.0, 3.0, 1.2, 1.6, 2.0, 3.0, 4.0))
g <- tda_graph(edges, directed = FALSE)
plot(g, width = 50, height = 70, layout = "tree", rt = 9, pl = 1, nc = 100)
pl = 2.
Box 8gd15b.cf
edges <- data.frame(
from = c(6, 6, 7, 7, 8, 8, 9, 9, 1, 1, 1, 7),
to = c(2, 4, 3, 5, 1, 6, 7, 8, 10, 11, 12, 13),
v = c(1.0, 1.5, 2.0, 2.5, 1.5, 1.0, 3.0, 1.2, 1.6, 2.0, 3.0, 4.0))
g <- tda_graph(edges, directed = FALSE)
plot(g, width = 90, height = 50, layout = "tree", rt = 9, pl = 2, nc = 100)
pltree lays a tree out by itself from a root;
plot(g, layout = "tree") is that command, and
gd15.cf’s tree of gd1.dat is the manual’s
figure.
Box 9pltree (gd15.cf)
3.4.2 Sequence Data Types
TDA reads sequence data two ways. Type 1 is one column per time
point. Type 2 reads the columns as state/time pairs, one pair per spell,
and builds the time axis from the times rather than from the column
count – so the same six columns become three variables over 0 to 8
instead of six over 0 to 5. type = 2 is TDA’s
seqdef(m = 2).
Box 10seq.d2 read both ways (seq3.cf)
seqd2 <- read.table(ex("seq.d2"),
col.names = c("ID", "Y1", "T1", "Y2", "T2", "Y3", "T3"))
tda_seq_info(seqd2[, -1]) sn type variables tmin tmax nstates states
1 1 1 6 0 5 9 0, 1, 2,....# seq3.cf declares both structures at once, the second as type 2; the
# manual's box shows them in one table, with their own time axes
seqd3i <- read.table(ex("seq.d3"),
col.names = c("ID", paste0("Y", 0:7),
"S1", "T1", "S2", "T2", "S3", "T3"))
tda_seq_info(list(paste0("Y", 0:7), c("S1", "T1", "S2", "T2", "S3", "T3")),
data = seqd3i, type = c(1, 2)) sn type variables tmin tmax nstates states
1 1 1 8 0 7 3 1, 3, 7
2 2 2 3 0 8 3 1, 2, 3tda_seqpd(seqd2, type = 2, id = "ID") case y0_0 y0_1 y0_2 y0_3 y0_4 y0_5 y0_6 y0_7 y0_8 id
1 1 1 1 1 1 -1 -1 -1 -1 -1 1
2 2 1 1 1 1 1 2 2 -1 -1 2
3 3 2 2 2 2 2 2 2 1 -1 3
4 4 1 1 1 1 1 1 2 -1 -1 4
5 5 -1 -1 -1 -1 1 1 2 2 3 53.4.3 Printing Sequence Data
seqpd writes the sequence data back out in one of four
layouts, TDA’s own m=: every structure side by side on one
row per case, one row per case and structure, one row per case and
position, or one row per spell with its origin and destination state and
the times it spans. The manual’s example defines two sequence structures
and so prints two sets of columns; this one has a single structure.
Box 11seqpd (seq5.cf)
# seq5.cf declares two structures over seq.d3: Y0..Y7 one column per
# time point, and three state/time pairs read as a type 2 sequence. The
# tables then carry a column per structure, and the time axis runs to 8
# rather than to 7.
seqd3 <- read.table(ex("seq.d3"),
col.names = c("ID", paste0("Y", 0:7),
"S1", "T1", "S2", "T2", "S3", "T3"))
pairs <- c("S1", "T1", "S2", "T2", "S3", "T3")
tda_seqpd(seqd3, id = "ID", variables = paste0("Y", 0:7), second = pairs) case y0_0 y0_1 y0_2 y0_3 y0_4 y0_5 y0_6 y0_7 y0_8 y1_0 y1_1 y1_2 y1_3 y1_4 y1_5 y1_6 y1_7 y1_8 id
1 1 7 3 3 1 3 3 7 7 -1 1 1 1 1 -1 -1 -1 -1 -1 1
2 2 3 3 7 1 1 -1 -1 3 -1 1 1 1 1 1 2 2 -1 -1 2
3 3 -1 3 3 -1 1 1 1 1 -1 2 2 2 2 2 2 2 1 -1 3
4 4 1 1 3 3 3 3 -1 -1 -1 1 1 1 1 1 1 2 -1 -1 4
5 5 7 3 3 1 3 3 7 7 -1 -1 -1 -1 -1 1 1 2 2 3 5tda_seqpd(seqd3, layout = 3, id = "ID", variables = paste0("Y", 0:7),
second = pairs) case time y0 y1 id
1 1 0 7 1 1
2 1 1 3 1 1
3 1 2 3 1 1
4 1 3 1 1 1
5 1 4 3 -1 1
6 1 5 3 -1 1
7 1 6 7 -1 1
8 1 7 7 -1 1
9 1 8 -1 -1 1
10 2 0 3 1 2
11 2 1 3 1 2
12 2 2 7 1 2
13 2 3 1 1 2
14 2 4 1 1 2
15 2 5 -1 2 2
16 2 6 -1 2 2
17 2 7 3 -1 2
18 2 8 -1 -1 2
19 3 0 -1 2 3
20 3 1 3 2 3
21 3 2 3 2 3
22 3 3 -1 2 3
23 3 4 1 2 3
24 3 5 1 2 3
25 3 6 1 2 3
26 3 7 1 1 3
27 3 8 -1 -1 3
28 4 0 1 1 4
29 4 1 1 1 4
30 4 2 3 1 4
31 4 3 3 1 4
32 4 4 3 1 4
33 4 5 3 1 4
34 4 6 -1 2 4
35 4 7 -1 -1 4
36 4 8 -1 -1 4
37 5 0 7 -1 5
38 5 1 3 -1 5
39 5 2 3 -1 5
40 5 3 1 -1 5
41 5 4 3 1 5
42 5 5 3 1 5
43 5 6 7 2 5
44 5 7 7 2 5
45 5 8 -1 3 5tda_seqpd(seqd3, layout = 4, id = "ID", variables = paste0("Y", 0:7),
second = pairs) case sn org des ts tf id
1 1 1 7 3 0 1 1
2 1 2 3 1 1 3 1
3 1 3 1 3 3 4 1
4 1 4 3 7 4 6 1
5 1 5 7 -1 6 8 1
6 1 6 -1 -1 8 9 1
7 2 1 3 7 0 2 2
8 2 2 7 1 2 3 2
9 2 3 1 -1 3 5 2
10 2 4 -1 3 5 7 2
11 2 5 3 -1 7 8 2
12 2 6 -1 -1 8 9 2
13 3 1 -1 3 0 1 3
14 3 2 3 -1 1 3 3
15 3 3 -1 1 3 4 3
16 3 4 1 -1 4 8 3
17 3 5 -1 -1 8 9 3
18 4 1 1 3 0 2 4
19 4 2 3 -1 2 6 4
20 4 3 -1 -1 6 9 4
21 5 1 7 3 0 1 5
22 5 2 3 1 1 3 5
23 5 3 1 3 3 4 5
24 5 4 3 7 4 6 5
25 5 5 7 -1 6 8 5
26 5 6 -1 -1 8 9 53.4.5 State Indicators
seqsi turns a sequence into an indicator matrix: for
each case and each time point asked for, one column per state, 1 where
the case is in that state. m = 2 counts how often each
state has been visited up to that point instead.
Box 12seqsi (seq6.cf)
seqd1i <- read.table(ex("seq.d1"))
names(seqd1i) <- c("ID", paste0("Y", seq_len(ncol(seqd1i) - 1L)))
tda_seqsi(seqd1i, tp = 1:5, id = "ID") case s1_1 s3_1 s7_1 s1_2 s3_2 s7_2 s1_3 s3_3 s7_3 s1_4 s3_4 s7_4 s1_5 s3_5 s7_5 id
1 1 0 1 0 0 1 0 1 0 0 0 1 0 0 1 0 1
2 2 0 1 0 0 0 1 1 0 0 1 0 0 0 0 0 2
3 3 0 1 0 0 1 0 0 0 0 1 0 0 1 0 0 3
4 4 1 0 0 0 1 0 0 1 0 0 1 0 0 1 0 4
5 5 0 1 0 0 1 0 1 0 0 0 1 0 0 1 0 5tda_seqsi(seqd1i, tp = 1:5, id = "ID", options = list(m = 2)) case s1_1 s3_1 s7_1 s1_2 s3_2 s7_2 s1_3 s3_3 s7_3 s1_4 s3_4 s7_4 s1_5 s3_5 s7_5 id
1 1 0 1 0 0 1 0 1 0 0 0 1 0 0 1 0 1
2 4 1 0 0 0 1 0 0 1 0 0 1 0 0 1 0 4
3 5 0 1 0 0 1 0 1 0 0 0 1 0 0 1 0 53.3.5 Joining Episode Data
ejoin merges two episode files onto one time axis: every
point where either file changes state starts a new sub-episode, and each
side’s state is carried into it (-3 where that side has
none). The manual’s own example is two files, six episodes and two,
joined into sixteen.
Box 13ejoin (ej1.cf)
ej1 <- read.table(ex("ej1.dat"),
col.names = c("ID", "NS", "SN", "TS", "TF", "S", "X1", "X2"))
ej2 <- read.table(ex("ej2.dat"),
col.names = c("ID", "NS", "SN", "TS", "TF", "S", "Y1"))
# every column past the structural six is carried through, filled -3
# where that side has no spell
tda_ejoin(ej1[, c("ID", "TS", "TF", "S", "X1", "X2")],
id = "ID", start = "TS", end = "TF", state = "S",
with = ej2[, c("ID", "TS", "TF", "S", "Y1")],
with_id = "ID", with_start = "TS", with_end = "TF",
with_state = "S",
options = list(nw = 3, fmt0 = 4, fmt1 = "8.4")) id nspell spellno level start end state1 state2 X1 X2 Y1
1 1 8 1 0 0 1 1 -3 1.1 11.1 -3.0
2 1 8 2 0 1 2 1 3 1.1 11.1 5.5
3 1 8 2 1 1 2 2 -3 2.2 22.2 -3.0
4 1 8 3 0 2 3 2 3 2.2 22.2 5.5
5 1 8 4 0 3 4 2 3 2.2 22.2 5.5
6 1 8 4 1 3 4 -3 4 -3.0 -3.0 6.6
7 1 8 5 0 4 5 2 4 2.2 22.2 6.6
8 1 8 6 0 5 6 -3 4 -3.0 -3.0 6.6
9 2 8 1 0 3 4 3 -3 3.3 33.3 -3.0
10 2 8 2 0 4 5 3 -3 3.3 33.3 -3.0
11 2 8 2 1 4 5 4 -3 4.4 44.4 -3.0
12 2 8 3 0 5 6 3 -3 3.3 33.3 -3.0
13 2 8 3 1 5 6 5 -3 5.5 55.5 -3.0
14 2 8 4 0 6 8 5 -3 5.5 55.5 -3.0
15 2 8 5 0 8 9 -3 -3 -3.0 -3.0 -3.0
16 2 8 6 0 9 10 6 -3 6.6 66.6 -3.04 PostScript Plots
TDA draws its PostScript. The command files of the manual’s chapter 4
are reproduced here through the tda_pl_* family; each box
names the command file it follows, and the package’s
inst/examples/gallery/ carries the same code as a
standalone script beside a PNG of TDA’s output for comparison.
4.1 Preparing the Plot Environment
The manual’s first plot: axes, a frame, the sine function and, by
plotf1, its first derivative – deriv = 1 in
tda_pl_function() draws the derivative of the expression it
is given, so the second call repeats sin(x1) as the command
file does.
Box 14plot1.cf
p <- tda_ps(xlim = c(0, 6), ylim = c(-1, 1), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(1, 1), ic = c(10, 0))
p <- tda_pl_frame(p)
p <- tda_pl_function(p, "sin(x1)", range = c(0, 6), step = 0.1)
p <- tda_pl_function(p, "sin(x1)", range = c(0, 6), step = 0.1, deriv = 1)
p <- tda_pl_text(p, "sine", at = c(2.7, 0.6))
p <- tda_pl_text(p, "cosine", at = c(4.6, 0.6))
plot(p)
Box 15plot2.cf
p <- tda_ps(xlim = c(0, 595), ylim = c(0, 842), width = 100, height = 150)
p <- tda_pl_axes(p, sc = 72)
p <- tda_pl_frame(p, gray = 0.95)
p <- tda_pl_grid(p, at_x = seq(72, 576, 72), at_y = seq(72, 792, 72))
plot(p)
4.2 Plotting Coordinate Systems
Box 16plot3.cf
p <- tda_ps(xlim = c(-2, 2), ylim = c(0, 1), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(1, 0.5), ic = c(2, 0))
p <- tda_pl_frame(p)
p <- tda_pl_grid(p, at_x = c(-1, 0, 1), at_y = 0.5)
p <- tda_pl_labels(p, "Linear Coordinate System", which = "title")
p <- tda_pl_labels(p, "X axis", which = "x")
p <- tda_pl_labels(p, "Y axis", which = "y", sc = 3)
plot(p)
Box 17plot4.cf
p <- tda_ps(xlim = c(1, 1000), ylim = c(2, 500), width = 90, height = 50,
log = "xy")
p <- tda_pl_axes(p, sc = c(10, 5), ic = c(10, 0))
p <- tda_pl_frame(p)
p <- tda_pl_grid(p, at_x = c(10, 100, 1000), at_y = c(10, 50, 250))
p <- tda_pl_labels(p, "Logarithmic Coordinate System", which = "title")
p <- tda_pl_labels(p, "X axis", which = "x")
p <- tda_pl_labels(p, "Y axis", which = "y", sc = 3)
plot(p)
A second pair of axes, drawn at a position given in user coordinates
rather than on the boundary of the plot: at = and
dir = in tda_pl_axis() are plx’s
and ply’s parameters.
Box 18plot5.cf
p <- tda_ps(xlim = c(-2, 2), ylim = c(0, 1), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(1, 0.5), ic = c(2, 0))
p <- tda_pl_axis(p, "x", sc = 1, ic = 5, dir = 1, at = c(-1, 1, 1, 1))
p <- tda_pl_axis(p, "y", sc = 0.5, ic = 5, dir = 1, at = c(2, 0, 2, 1))
plot(p)
A second panel on the same page, placed with
tda_pl_panel().
Box 19plot6.cf
# plot6.cf places the first plot with psorg = 100,600, in PostScript
# points; origin = is in mm like width = and height =
p <- tda_ps(xlim = c(0, 6), ylim = c(-1, 1), width = 50, height = 40,
origin = c(100, 600) * 25.4 / 72)
p <- tda_pl_axes(p, sc = 1)
p <- tda_pl_function(p, "sin(x1)", range = c(0, 6), step = 0.1)
p <- tda_pl_text(p, "Plot 1", at = c(4, 0.2))
# the second plot has psorg = 270,600: 170 points to the right of the
# first, and a panel's origin is relative to the previous one's
p <- tda_pl_panel(p, origin = c(170, 0) * 25.4 / 72, width = 50, height = 30)
p <- tda_pl_axes(p, sc = 1)
p <- tda_pl_function(p, "sin(x1)", range = c(0, 6), step = 0.1)
p <- tda_pl_text(p, "Plot 2", at = c(4, 0.2))
plot(p)
4.3 Combining Plot Files
dplot puts PostScript files that already exist into a
grid. The two coordinate systems of 4.2 are drawn again as files, and
tda_combine_ps() places each of them twice on one page.
Box 20plot7.cf
p3 <- tda_ps(xlim = c(-2, 2), ylim = c(0, 1), width = 90, height = 50)
p3 <- tda_pl_axes(p3, sc = c(1, 0.5), ic = c(2, 0))
p3 <- tda_pl_frame(p3)
p3 <- tda_pl_grid(p3, at_x = c(-1, 0, 1), at_y = 0.5)
p3 <- tda_pl_labels(p3, "Linear Coordinate System", which = "title")
p3 <- tda_pl_labels(p3, "X axis", which = "x")
p3 <- tda_pl_labels(p3, "Y axis", which = "y", sc = 3)
f3 <- tda_ps_file(p3)
p4 <- tda_ps(xlim = c(1, 1000), ylim = c(2, 500), width = 90, height = 50,
log = "xy")
p4 <- tda_pl_axes(p4, sc = c(10, 5), ic = c(10, 0))
p4 <- tda_pl_frame(p4)
p4 <- tda_pl_grid(p4, at_x = c(10, 100, 1000), at_y = c(10, 50, 250))
p4 <- tda_pl_labels(p4, "Logarithmic Coordinate System", which = "title")
p4 <- tda_pl_labels(p4, "X axis", which = "x")
p4 <- tda_pl_labels(p4, "Y axis", which = "y", sc = 3)
f4 <- tda_ps_file(p4)
r <- tda_combine_ps(list(c(f3, f3), c(f4, f4)), width = 100, height = 60,
file = "plot7.ps")
tda_plot_ps(file.path(r$dir, "plot7.ps"))
4.4.1 Polygons and Scatterplots
plot8.cf draws twenty points with
rd(-2, 2), sorts them, and takes nd() of each
– TDA’s standard normal distribution function. Both are used here, so
the points are the manual’s.
Box 21plot8.cf
pdat <- tda_derive(data.frame(RDS = sort(tda_rd(20, -2, 2))),
ND = "nd(RDS)")
p <- tda_ps(pdat, xlim = c(-2, 2), ylim = c(0, 1), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(0.5, 1), ic = c(0, 10), fmt = c("4.1", NA))
p <- tda_pl_frame(p)
p <- tda_pl_points(p, "RDS", "ND", symbol = 4, size = 1.5, lty = 1)
plot(p)
The nine line types, lty = 1 to 9.
Box 22plot9.cf
p <- tda_ps(xlim = c(-2, 2), ylim = c(0, 1), width = 100, height = 60)
p <- tda_pl_frame(p)
for (lt in 1:9)
p <- tda_pl_lines(p, x = c(-1.5, 0.5), y = c(lt, lt) / 10, lty = lt)
for (lt in 1:9)
p <- tda_pl_text(p, sprintf("line type %d", lt), at = c(0.75, lt / 10))
plot(p)
The seventeen marker symbols: symbol = in
tda_pl_text() is pltext’s s=, and
puts the marker at the text position.
Box 23plot10.cf
p <- tda_ps(xlim = c(-2, 2), ylim = c(0, 1), width = 100, height = 60)
p <- tda_pl_frame(p)
for (s in 9:1)
p <- tda_pl_text(p, sprintf("marker %d", s), at = c(-1.5, (10 - s) / 10),
symbol = s)
for (s in 17:10)
p <- tda_pl_text(p, sprintf("marker %d", s), at = c(0, (19 - s) / 10),
symbol = s)
plot(p)
4.4.3 Plotting Text
TDA writes special characters as @nnn (the Symbol font)
or \\nnn (Times-Roman, through its own re-encoding table);
both round-trip, a literal UTF-8 character does not.
plots.cf shows each escape and, beside it, the escape
written out literally: \\100 is the @ sign and
\\\\ a backslash, so \\100141 prints as
@141, and \\324 is the quotation mark around
each. The R strings below carry one more level of backslashes than the
command file, R’s own.
Box 24plots.cf
p <- tda_ps(xlim = c(0, 10), ylim = c(0, 5), width = 120, height = 60)
p <- tda_pl_axes(p, sc = 1, ic = c(0, 2))
p <- tda_pl_frame(p)
p <- tda_pl_text(p, "@141 = 2.5 \\245 @142", at = c(1, 4))
p <- tda_pl_text(p, "pltext (xy=1,4) = \\324\\100141 = 2.5 \\\\245 \\100142\\324",
at = c(4, 4))
p <- tda_pl_text(p, "@123 = @173 @161 @174 @161 @316 @121 @175", at = c(1, 2))
p <- tda_pl_text(p, paste("pltext (xy=1,2) = \\324\\100123 = \\100173 \\100161",
"\\100174 \\100161 \\100316 \\100121 \\100175\\324"),
at = c(2.2, 0.9))
p <- tda_pl_text(p, "Sm\\277rebr\\277d", at = c(1, 3))
p <- tda_pl_text(p, "pltext (xy=1,3) = \\324Sm\\\\277rebr\\\\277d\\324", at = c(4, 3))
p <- tda_pl_lines(p, x = c(3.5, 2.7), y = c(4.05, 4.05), a = "1,1")
p <- tda_pl_lines(p, x = c(3.5, 2.7), y = c(3.05, 3.05), a = "1,1")
p <- tda_pl_arc(p, x = c(1.5, 3.2, 3.2), y = c(1.5, 1.5, 2.1), curvature = -90)
p <- tda_pl_lines(p, x = c(3.2, 2.9), y = c(2.1, 2.1), a = "1,1")
p <- tda_pl_arc(p, x = c(1.5, 1.5), y = c(1.5, 0.95), curvature = -90)
p <- tda_pl_lines(p, x = c(1.5, 2), y = c(0.95, 0.95))
plot(p)
4.4.5 Step Functions
plot11.cf is plot8.cf with
dir=1 on the plot command: the pen moves along
the x axis first and then along y, and the twenty points on the normal
distribution function become a step function. dir keeps its
name in tda_pl_lines().
Box 25plot11.cf
pdat <- tda_derive(data.frame(RDS = sort(tda_rd(20, -2, 2))),
ND = "nd(RDS)")
p <- tda_ps(pdat, xlim = c(-2, 2), ylim = c(0, 1), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(0.5, 1), ic = c(0, 10), fmt = c("4.1", NA))
p <- tda_pl_frame(p)
p <- tda_pl_lines(p, "RDS", "ND", dir = 1, gray = 0.9)
plot(p)
4.4.6 General Functions
Two normal densities, one of them filled, and a line at each mean.
The expression string goes to TDA’s parser, not R’s, so it is written
with TDA’s functions – here ndf(), the standard normal
density.
Box 26plot12.cf
p <- tda_ps(xlim = c(-3, 3), ylim = c(0, 1), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(0.5, 1), ic = c(0, 10), fmt = c("4.1", NA))
p <- tda_pl_frame(p)
p <- tda_pl_function(p, "ndf((x1 - 1.5) / 0.5) / 0.5", range = c(0, 3),
step = 0.05)
p <- tda_pl_lines(p, x = c(1.5, 1.5), y = c(0, 0.798))
p <- tda_pl_function(p, "ndf(x1)", range = c(-3, 3), step = 0.1, gray = 0.8)
p <- tda_pl_lines(p, x = c(0, 0), y = c(0, 0.399))
plot(p)
4.4.7 Arrows
A polygon can end in an arrow: a = xs, ys gives the
length and width of the head in millimetres. plot13.cf
draws eight of them from one point, in two line types and with heads of
growing size; a keeps its name in
tda_pl_lines().
Box 27plot13.cf
p <- tda_ps(xlim = c(0, 10), ylim = c(0, 6), width = 90, height = 50)
p <- tda_pl_axes(p, sc = 1)
p <- tda_pl_frame(p)
p <- tda_pl_lines(p, x = c(5, 8), y = c(3, 3), a = "1,1")
p <- tda_pl_lines(p, x = c(5, 7), y = c(3, 5), a = "1.5,1.5")
p <- tda_pl_lines(p, x = c(5, 5), y = c(3, 5.5), a = "2,2")
p <- tda_pl_lines(p, x = c(5, 3), y = c(3, 5), a = "2.5,2.5")
p <- tda_pl_lines(p, x = c(5, 2), y = c(3, 3), a = "1,1", lty = 5)
p <- tda_pl_lines(p, x = c(5, 3), y = c(3, 1), a = "1.5,1.5", lty = 5)
p <- tda_pl_lines(p, x = c(5, 5), y = c(3, 0.5), a = "2,2", lty = 5)
p <- tda_pl_lines(p, x = c(5, 7), y = c(3, 1), a = "2.5,2.5", lty = 5)
p <- tda_pl_text(p, "[1,1]", at = c(8.3, 3))
p <- tda_pl_text(p, "[1.5,1.5]", at = c(7.3, 5))
p <- tda_pl_text(p, "[2,2]", at = c(5.3, 5.6))
p <- tda_pl_text(p, "[2.5,2.5]", at = c(1.6, 5))
plot(p)
4.4.8 Circles, Arcs, Ellipses
ploto, which is tda_pl_circle(), draws a
circle or, with angles =, an arc of one;
plotk, which is tda_pl_arc(), connects points
by circular arcs that can carry an arrowhead. The manual’s Figure 3
labels one of them with the Symbol font’s alpha, which TDA writes as
@141; the same escape is used here (see 4.4.3).
Box 28plot14.cf
p <- tda_ps(xlim = c(0, 10), ylim = c(0, 6), width = 90, height = 50)
p <- tda_pl_axes(p, sc = 1)
p <- tda_pl_frame(p)
p <- tda_pl_lines(p, x = c(1, 4), y = c(2, 5), lty = 5)
p <- tda_pl_arc(p, x = c(1, 4), y = c(2, 5), curvature = 45, arrow = c(1, 1))
p <- tda_pl_lines(p, x = c(1, 1), y = c(2, 4), lty = 5)
p <- tda_pl_circle(p, at = c(1, 2), r = 10, angles = c(45, 90))
p <- tda_pl_text(p, "plotk(a=1,1,sc=45) = a,b,c,d;", at = c(0.5, 1))
p <- tda_pl_text(p, "(a,b)", at = c(0.7, 1.6))
p <- tda_pl_text(p, "(c,d)", at = c(4.2, 5.1))
p <- tda_pl_circle(p, at = c(1, 2), r = 0.8, angles = c(45, 90), lty = 2)
p <- tda_pl_text(p, "@141", at = c(1.15, 2.4))
p <- tda_pl_arc(p, x = c(6, 6, 7), y = c(2, 4, 5), curvature = 45)
p <- tda_pl_arc(p, x = c(9, 9, 8), y = c(2, 4, 5), curvature = -45)
p <- tda_pl_text(p, "plotk(sc=45) = 6,2,6,4,7,5;", at = c(5.5, 1.3))
p <- tda_pl_text(p, "plotk(sc=-45) = 9,2,9,4,8,5;", at = c(5.5, 0.8))
p <- tda_pl_lines(p, x = c(5.3, 5.3, 5.7), y = c(1.4, 2.2, 2.5), a = "1,1")
p <- tda_pl_lines(p, x = c(9.6, 9.6, 9.2), y = c(1.2, 2.2, 2.5), a = "1,1")
plot(p)
A shaded circle and ellipse, dimension lines, and labelled arrows.
Box 29plot15.cf
p <- tda_ps(xlim = c(0, 10), ylim = c(0, 2), width = 90, height = 50)
p <- tda_pl_axes(p, sc = 1)
p <- tda_pl_frame(p)
p <- tda_pl_circle(p, at = c(2, 1), r = 1, gray = 0.9)
p <- tda_pl_ellipse(p, at = c(7, 1), axes = c(2, 0.5), gray = 0.9)
p <- tda_pl_lines(p, x = c(0, 7), y = c(0.5, 0.5), lty = 5)
p <- tda_pl_lines(p, x = c(0, 7), y = c(1.5, 1.5), lty = 5)
p <- tda_pl_lines(p, x = c(1, 1), y = c(0, 1), lty = 5)
p <- tda_pl_lines(p, x = c(3, 3), y = c(0, 1), lty = 5)
p <- tda_pl_lines(p, x = c(5, 5), y = c(0, 1), lty = 5)
p <- tda_pl_lines(p, x = c(9, 9), y = c(0, 1), lty = 5)
p <- tda_pl_text(p, "ploto (xy=2,1,gs=0.9) = 1;", at = c(1.1, 1.7))
p <- tda_pl_text(p, "plote (xy=7,1,gs=0.9) = 2,0.5;", at = c(5.1, 1.7))
p <- tda_pl_lines(p, x = c(2, 3), y = c(1, 1), a = "1,1")
p <- tda_pl_lines(p, x = c(7, 9), y = c(1, 1), a = "1,1")
p <- tda_pl_lines(p, x = c(7, 7), y = c(1, 1.5), a = "1,1")
p <- tda_pl_text(p, "1", at = c(2.3, 0.85))
p <- tda_pl_text(p, "2", at = c(7.8, 0.85))
p <- tda_pl_text(p, "0.5", at = c(7.2, 1.2))
plot(p)
Three ellipses at one centre, two of them rotated:
rotate = in tda_pl_ellipse() is the third
value on plote’s right-hand side, the angle in degrees.
Box 30plot16.cf
p <- tda_ps(xlim = c(0, 10), ylim = c(0, 2), width = 90, height = 50)
p <- tda_pl_axes(p, sc = 1)
p <- tda_pl_ellipse(p, at = c(4, 1), axes = c(3, 0.3), gray = 0.9)
p <- tda_pl_ellipse(p, at = c(4, 1), axes = c(3, 0.3), rotate = 45)
p <- tda_pl_ellipse(p, at = c(4, 1), axes = c(3, 0.3), rotate = -45)
p <- tda_pl_text(p, "plote(xy=4,1,gs=0.9) = 3,0.3;", at = c(7.3, 1.0))
p <- tda_pl_text(p, "plote(xy=4,1) = 3,0.3,-45;", at = c(7.3, 0.5))
p <- tda_pl_text(p, "plote(xy=4,1) = 3,0.3,45;", at = c(7.3, 1.5))
p <- tda_pl_rect(p, x = c(11.5, 11.5), y = c(0, 0))
plot(p)
4.4.9 Convex Hull
plot17.cf draws one hundred points from
X = rd, Y = rd and puts a convex hull around them. The
draws come from TDA’s generator at its default seed, so these are the
manual’s points: the PostScript this writes carries the same drawing
operations, in the same order, as TDA’s own run of the command file.
Box 31plot17.cf
gen <- tda_rng()
pdat <- data.frame(X = numeric(100), Y = numeric(100))
for (i in 1:100) {
pdat$X[i] <- gen$rd()
pdat$Y[i] <- gen$rd()
}
p <- tda_ps(pdat, xlim = c(-0.5, 1.5), ylim = c(-0.5, 1.5),
width = 90, height = 50)
p <- tda_pl_axes(p, sc = 0.5)
p <- tda_pl_frame(p)
plot(tda_pl_hull(p, "X", "Y", gray = 0.95, symbol = 5, size = 0.7))
plot18.cf smooths the same hull. smooth is
plotch’s ns=, the number of intervals for its
Akima smoothing, and expand is ic=, a margin
in millimetres: each point of the smoothed outline is pushed away from
the hull’s centroid in proportion to its distance from it, so the
outline grows without changing shape.
Box 32plot18.cf
plot(tda_pl_hull(p, "X", "Y", gray = 0.95, symbol = 5, size = 0.7,
smooth = 10, expand = 2))
4.4.10 Smoothing Polygons
plots draws an Akima spline through the points of two
variables, and plotsp the same through literal coordinates;
in the C both reach the same function, as plot and
plotp do. tda_pl_smooth() is
plots, and the points are given as a data frame.
Box 33plot19.cf
pdat <- data.frame(x = c(1, 2, 3, 4, 5, 6), y = c(1, 2.5, 2, 2.5, 2, 1))
p <- tda_ps(pdat, xlim = c(0, 7), ylim = c(0, 3), width = 90, height = 50)
p <- tda_pl_axes(p, sc = 1)
p <- tda_pl_frame(p)
p <- tda_pl_smooth(p, "x", "y", ns = 10, gray = 0.9)
p <- tda_pl_points(p, "x", "y", symbol = 5)
plot(p)
plot20.cf closes the same curve: sc = 1,
which keeps its name.
Box 34plot20.cf
pdat <- data.frame(x = c(1, 2, 3, 4, 5, 6), y = c(1, 2.5, 2, 2.5, 2, 1))
p <- tda_ps(pdat, xlim = c(0, 7), ylim = c(0, 3), width = 90, height = 50)
p <- tda_pl_axes(p, sc = 1)
p <- tda_pl_frame(p)
p <- tda_pl_smooth(p, "x", "y", ns = 10, gray = 0.9, sc = 1)
p <- tda_pl_points(p, "x", "y", symbol = 5)
plot(p)
4.4.11 Contour Plots
plotc draws contour lines of a function of two
variables, x1 and x2 in TDA’s expression
syntax, over a grid of the given resolution; plot21.cf’s
function is the sum of two normal densities.
Box 35plot21.cf
p <- tda_ps(xlim = c(-2, 2), ylim = c(-2, 2), width = 70, height = 70)
p <- tda_pl_axes(p, sc = 1)
p <- tda_pl_frame(p)
p <- tda_pl_contour(p,
"exp(-((x1+1)^2+(x2+1)^2)/2)/(2*pi)+exp(-((x1-1)^2+(x2-1)^2)/2)/(2*pi)",
levels = c(0.01, 0.05, 0.1, 0.125, 0.15), resolution = c(50, 50),
gray = 0.9)
plot(p)
5 Mathematical Procedures
5.1.2 Matrices
TDA has its matrix language, and a fair part of it exists because TDA has no other way to get a matrix into a session. R does, so those commands have no wrapper and need none:
| TDA | R |
|---|---|
mdef (a literal matrix) |
matrix() |
mdefc (constant) |
matrix(d, m, n) |
mdefi (rectangular identity) |
diag(1, m, n) |
mdeff (from a file) |
as.matrix(read.table(...)) |
mpr (print an expression) |
printing the return value |
mplog, mppar, mpcov,
mpgrad
|
logLik(), $estimates,
vcov()
|
Two do compute something and do have wrappers: mdefg, a
graph’s adjacency matrix, is tda_mdefg(), and
mbrr is tda_brr(). mdefb extracts
a block of the data matrix and is reached through the
commands= argument of tda_dblock(). Everything
else in 5.1.4 takes and returns a plain R matrix.
The manual’s matrix chapter prints with mfmt = 13.10;
these boxes print with ten digits too, so a number can be read off
against it.
5.1.4.1 Elementary Matrix Commands
Box 36Shape, product, transpose, trace
op <- options(digits = 10)
A <- matrix(1:6, 3, 2)
c(tda_mnrow(A), tda_mncol(A))[1] 3 2tda_mmul(A, tda_mtransp(A)) [,1] [,2] [,3]
[1,] 17 22 27
[2,] 22 29 36
[3,] 27 36 45tda_mtrace(tda_mcross(A))[1] 91mcross is t(A) %*% A, and
mdiag, mdrow, mdcol and
mdiagd make or take diagonals.
Box 37Diagonals
tda_mdiag(1:4) [,1] [,2] [,3] [,4]
[1,] 1 0 0 0
[2,] 0 2 0 0
[3,] 0 0 3 0
[4,] 0 0 0 4tda_mdrow(A) [,1] [,2] [,3]
[1,] 5 0 0
[2,] 0 7 0
[3,] 0 0 9tda_mdcol(A) [,1] [,2]
[1,] 6 0
[2,] 0 15tda_mdiagd(A)[1] 1 5msqrtd and msqrti are the elementwise
square root and its inverse.
Box 38Elementwise square roots
tda_msqrtd(matrix(1:4, 2, 2)) [,1] [,2]
[1,] 1 0
[2,] 0 2tda_msqrti(matrix(1:4, 2, 2)) [,1] [,2]
[1,] 1 0.0
[2,] 0 0.5Concatenation, row and column sums, selection and trimming.
Box 39Concatenating, summing, selecting, trimming
tda_mcath(A, A) [,1] [,2] [,3] [,4]
[1,] 1 4 1 4
[2,] 2 5 2 5
[3,] 3 6 3 6tda_mcatv(A, A) [,1] [,2]
[1,] 1 4
[2,] 2 5
[3,] 3 6
[4,] 1 4
[5,] 2 5
[6,] 3 6tda_mrsum(A)[1] 5 7 9tda_mcsum(A)[1] 6 15tda_msrow(A, 1:2) [,1] [,2]
[1,] 1 4
[2,] 2 5tda_mscol(A, 1) [,1]
[1,] 1
[2,] 2
[3,] 3tda_mtrim(A, 1) [,1]
[1,] 4
[2,] 5
[3,] 6tda_mtrim(A, leading_rows = 1) [,1] [,2]
[1,] 2 5
[2,] 3 6Vectorisation, the Kronecker product, and the permutation commands.
mpinv inverts a permutation rather than a matrix.
Box 40Vectorising, Kronecker product, permutations
tda_mcvec(A)[1] 1 2 3 4 5 6tda_mrvec(A)[1] 1 4 2 5 3 6tda_mkp(A, A) [,1] [,2] [,3] [,4]
[1,] 1 4 4 16
[2,] 2 5 8 20
[3,] 3 6 12 24
[4,] 2 8 5 20
[5,] 4 10 10 25
[6,] 6 12 15 30
[7,] 3 12 6 24
[8,] 6 15 12 30
[9,] 9 18 18 36tda_mprow(A, c(2, 1, 3)) [,1] [,2]
[1,] 2 5
[2,] 1 4
[3,] 3 6tda_mpcol(A, c(2, 1)) [,1] [,2]
[1,] 4 1
[2,] 5 2
[3,] 6 3tda_mpsym(matrix(1:4, 2, 2), c(2, 1)) [,1] [,2]
[1,] 4 2
[2,] 3 1tda_mpinv(c(4, 2, 1, 3))[1] 3 2 4 1mag solves a system by Gaussian elimination, and reports
what it found when there is no solution or more than one.
Box 41Gaussian elimination
5.1.4.2 Sorting and Ranking
msort sorts the rows of a matrix by a column,
msort1 a single column, and mrank ranks.
Box 42msort, msort1, mrank
5.1.4.3 Standardization and Scaling
Box 43mcent, mstand, mscal1
tda_mcent(A) [,1] [,2]
[1,] -1 -1
[2,] 0 0
[3,] 1 1tda_mstand(A) [,1] [,2]
[1,] -1.224744871 -1.224744871
[2,] 0.000000000 0.000000000
[3,] 1.224744871 1.224744871tda_mscal1(A) [,1] [,2]
[1,] 0.04761904762 0.1904761905
[2,] 0.09523809524 0.2380952381
[3,] 0.14285714286 0.28571428575.1.4.4 Matrix Norms
Box 44The three norms
c(tda_mnorm(A), tda_mnorm1(A), tda_mnorm2(A))[1] 6.000000000 21.000000000 9.5393920145.1.4.5 Inverse Matrices
minvd and minvs invert a general and a
symmetric matrix. mginv is the generalized inverse, and
what makes it one is that A B A is A again –
which is the manual’s check.
Box 45minvd, minvs
A [,1] [,2]
[1,] 1 4
[2,] 2 5
[3,] 3 6tda_minvd(A) [,1] [,2]
[1,] 1 0.0
[2,] 0 0.2
[3,] 0 0.0diag(1:2) [,1] [,2]
[1,] 1 0
[2,] 0 2tda_minvs(diag(1:2)) [,1] [,2]
[1,] 1 0.0
[2,] 0 0.5Box 46A generalized inverse, and the identity that defines it
B0 <- matrix(c(1, 5, 0, 2, 6, 0, 3, 7, 0, 4, 8, 0), 4, 3, byrow = TRUE)
Bg <- tda_mginv(B0)
Bg [,1] [,2] [,3] [,4]
[1,] -0.55 -0.225 1.000000000e-01 0.425
[2,] 0.25 0.125 4.857225733e-17 -0.125
[3,] 0.00 0.000 0.000000000e+00 0.000tda_mmul(tda_mmul(B0, Bg), B0) [,1] [,2] [,3]
[1,] 1 5 0
[2,] 2 6 0
[3,] 3 7 0
[4,] 4 8 05.1.4.7 Least Squares Regression
mls takes the design and the response in one matrix, the
response last.
Box 48mls
5.1.4.8 Linear Equations
Box 49mlse
mlseA <- matrix(c(-0.4744, -0.4993, -0.7250, -0.8615, 0.5840, -0.7947,
0.1652, 0.0, -0.6587, -0.3450, 0.6687, 0.0), 3, 4,
byrow = TRUE)
mlseA # the manual's box prints the input too [,1] [,2] [,3] [,4]
[1,] -0.4744 -0.4993 -0.7250 -0.8615
[2,] 0.5840 -0.7947 0.1652 0.0000
[3,] -0.6587 -0.3450 0.6687 0.0000tda_mlse(mlseA)$x
[1] 0.4087254230 0.4301912621 0.6245602237
$residual
[1] 3.705655206e-16
$rank
[1] 35.1.4.11 Eigenvalues and Eigenvectors
mev is the general case, which may have complex
eigenvalues, and mevs the symmetric one. The manual’s
mev example is a defective problem – 5.236 and 0.764 are
each a double eigenvalue – so the digits beyond the seventh, and which
two vectors span each repeated eigenvalue’s space, depend on the
machine’s arithmetic; the manual’s box was computed on a different one
and differs there, not in the eigenvalues themselves.
Box 51mev and mevs
tda_mev(matrix(c(6, -3, 4, 1, 4, 2, 4, 0, 4, -2, 3, 1, 4, 2, 3, 1),
4, 4, byrow = TRUE))$values
[1] 5.2360678384+0i 5.2360681166+0i 0.7639319802+0i 0.7639320648+0i
$vectors
[,1] [,2] [,3] [,4]
[1,] -0.2627643793+0i -0.2627644056+0i 0.3311926976+0i 0.3311926619+0i
[2,] -0.6152998198+0i -0.6152998173+0i -0.1131489151+0i -0.1131489401+0i
[3,] -0.2350236102+0i -0.2350236246+0i -0.2962277587+0i -0.2962277177+0i
[4,] -0.7050708584+0i -0.7050708460+0i -0.8886832028+0i -0.8886832266+0i
$er
[1] 5.2360678384 5.2360681166 0.7639319802 0.7639320648
$ei
[1] 0 0 0 0
$evr
[,1] [,2] [,3] [,4]
[1,] -0.2627643793 -0.2627644056 0.3311926976 0.3311926619
[2,] -0.6152998198 -0.6152998173 -0.1131489151 -0.1131489401
[3,] -0.2350236102 -0.2350236246 -0.2962277587 -0.2962277177
[4,] -0.7050708584 -0.7050708460 -0.8886832028 -0.8886832266
$evi
[,1] [,2] [,3] [,4]
[1,] 0 0 0 0
[2,] 0 0 0 0
[3,] 0 0 0 0
[4,] 0 0 0 0tda_mevs(matrix(c(1, 1, 1, 1, 2, 0, 1, 0, 10), 3, 3, byrow = TRUE))$values
[1] 10.1112597744 2.5823542359 0.3063859897
$vectors
[,1] [,2] [,3]
[1,] 0.11056720082 -0.50208545527 -0.85772086934
[2,] 0.01363132286 -0.86216502656 0.50644412724
[3,] 0.99377516629 0.06768797962 0.088483084685.1.4.12 Singular Value Decomposition
On a cross-product matrix the singular values are the eigenvalues,
which is the manual’s way of showing the two agree. msvd1
prints as in the manual’s box; mevs returns its eigenvalues
in descending order where the manual’s box (2002) shows them ascending –
TDA 6.4p’s evectf() sorts descending, so that is the
program’s answer, not the manual’s.
Box 52msvd1 against mevs
A <- matrix(c(1, 0, 0, 5, 7, 0, 1, 0, 1), 3, 3, byrow = TRUE)
S <- tda_mcross(A)
S # the cross-product matrix the manual's box prints [,1] [,2] [,3]
[1,] 27 35 1
[2,] 35 49 0
[3,] 1 0 1tda_msvd1(S)$d
[1] 74.6926238609 1.9752562176 0.3321199215
$u
[,1] [,2] [,3]
[1,] -0.591733290081 0.6195362372 0.5157776305
[2,] -0.806093813732 -0.4611140127 -0.3709267188
[3,] -0.008029749235 0.6352548449 -0.7722608401
$v
[,1] [,2] [,3]
[1,] -0.591733290081 0.6195362372 0.5157776305
[2,] -0.806093813732 -0.4611140127 -0.3709267188
[3,] -0.008029749235 0.6352548449 -0.7722608401tda_mevs(S)$values
[1] 74.6926238609 1.9752562176 0.3321199215
$vectors
[,1] [,2] [,3]
[1,] -0.591733290081 -0.6195362372 -0.5157776305
[2,] -0.806093813732 0.4611140127 0.3709267188
[3,] -0.008029749235 -0.6352548449 0.77226084015.1.4.13 Linear Programming
The manual’s example is mlp(T,X,Y,1): maximise x1
subject to x1 <= 1 and one equality, x2 = 1, which is the last row of
T. That is tda_mlp1(), with the equality given as its
row.
Box 53mlp with one equality row
# the manual writes the whole problem as one matrix T: the objective in
# the first row, then the inequality, then the equality
Tlp <- matrix(c(1, 0, 0,
1, 0, 1,
0, 1, 1), 3, 3, byrow = TRUE)
Tlp [,1] [,2] [,3]
[1,] 1 0 0
[2,] 1 0 1
[3,] 0 1 1tda_mlp1(objective = c(1, 0), constraints = matrix(c(1, 0), 1), bounds = 1,
equalities = matrix(c(0, 1), 1), equalities_bounds = 1)$value
[1] 1
$x
[1] 1 1
$y
[1] 1 05.1.4.14 Some Further Commands
mproc is a Procrustes rotation: the rotation, scaling
and translation that brings one configuration onto another. The two
squares below differ by a factor of two, and that is what it
recovers.
Box 54mproc, mnum
5.2.2 Evaluating Expressions
tda_evalf() evaluates an expression in TDA’s language,
with the variables given as options.
Box 55evalf
5.2.4 Random Numbers
tda_rng() is TDA’s generator – multiplicative
congruential, with a fixed seed – so a sequence drawn from it is the
sequence TDA drew. rd() is uniform, rdn()
normal. tda_runif() is the same generator as a plain
vector.
Box 56TDA’s random numbers
5.2.6.6 Counting Changes
The operators of the expression language that have no R counterpart
are reached with tda_derive(): it runs nvar on
a data frame with new variables given as TDA expressions and returns the
frame with them added. The manual’s example for change and
cntch is a small sequence of states per person, in record
mode and then in block mode (block = "ID", TDA’s
dblock=), where the counts are per person.
Box 57change and cntch, record mode and block mode
seqs <- data.frame(ID = c(1, 1, 1, 2, 2, 3, 3, 3, 3, 3, 3),
S = c(1, 3, 2, 2, 3, 3, 1, 2, 2, 2, 1),
T = c(10, 15, 20, 12, 18, 8, 12, 15, 17, 18, 19))
# the manual's five: whether the state changed, how many changes in
# all, the running count, how far back the last change was, and the time
# at that change
tda_derive(seqs, CH = "change(S)", CNT = "cntch(S)", CCNT = "ccntch(S)",
LAGCH = "lagch(S)", LAGT = "lag(T,-lagch(S))") ID S T CH CNT CCNT LAGCH LAGT
1 1 1 10 1 7 1 0 10
2 1 3 15 1 7 2 0 15
3 1 2 20 1 7 3 0 20
4 2 2 12 0 7 3 1 20
5 2 3 18 1 7 4 0 18
6 3 3 8 0 7 4 1 18
7 3 1 12 1 7 5 0 12
8 3 2 15 1 7 6 0 15
9 3 2 17 0 7 6 1 15
10 3 2 18 0 7 6 2 15
11 3 1 19 1 7 7 0 19tda_derive(seqs, CH = "change(S)", CNT = "cntch(S)", CCNT = "ccntch(S)",
LAGCH = "lagch(S)", LAGT = "lag(T,-lagch(S))", block = "ID") ID S T CH CNT CCNT LAGCH LAGT
1 1 1 10 1 3 1 0 10
2 1 3 15 1 3 2 0 15
3 1 2 20 1 3 3 0 20
4 2 2 12 1 2 1 0 12
5 2 3 18 1 2 2 0 18
6 3 3 8 1 4 1 0 8
7 3 1 12 1 4 2 0 12
8 3 2 15 1 4 3 0 15
9 3 2 17 0 4 3 1 15
10 3 2 18 0 4 3 2 15
11 3 1 19 1 4 4 0 195.2.6.7 Aggregates
The g operators aggregate within blocks:
gcnt counts the block, grec numbers the record
within it, gmean and gstd average a variable
over it.
Box 58gcnt, grec, gmean, gstd
agg <- data.frame(V = c(1, 4, 5, 3, 6, 2), A = c(1, 1, 1, 2, 7, 7))
# the manual's five: the block's size, the record's number in it,
# the block's number, and whether the record is its first or last
tda_derive(agg, GCNT = "gcnt(A)", GREC = "grec(A)", GSN = "gsn(A)",
GFIRST = "gfirst(A)", GLAST = "glast(A)") V A GCNT GREC GSN GFIRST GLAST
1 1 1 3 1 1 1 0
2 4 1 3 2 1 0 0
3 5 1 3 3 1 0 1
4 3 2 1 1 2 1 1
5 6 7 2 1 3 1 0
6 2 7 2 2 3 0 1# gmean and gstd average a variable over the block
tda_derive(agg, GM = "gmean(V,A)", GS = "gstd(V,A)", block = "A") V A GM GS
1 1 1 3.333333 2.081666
2 4 1 3.333333 2.081666
3 5 1 3.333333 2.081666
4 3 2 3.000000 0.000000
5 6 7 4.000000 2.828427
6 2 7 4.000000 2.8284275.3.3 Evaluating Functions
tda_evalf() evaluates a function;
derivatives = 2 is evalf2, the value with the
gradient and the Hessian, which TDA computes analytically – the manual’s
deriv1.cf.
Box 59evalf2: a function, its gradient and Hessian (deriv1.cf)
5.4.1 Commands for Numerical Integration
int1.cf integrates sqrt(x) over 0..1 with the QNG
method, which should give 2/3; the manual’s box shows the approximation
and the number of function calls.
Box 60integrate: int1.cf
int1 <- tda_integrate("sqrt(x)", from = 0, to = 1)
int1$value[1] 0.6666669# the rest of what TDA reports: the rule, the relative error it worked
# to, and how many times it called the function
c(method = int1$method, rel_error = int1$rel_error, calls = int1$calls) method rel_error calls
"1 (QNG)." "1e-04" "43" 5.4.2 Operators for Numerical Integration
int(a, b, f) is the integral as an operator inside an
expression, so its limits may be expressions themselves;
int2.cf and int3.cf.
Box 61int() inside an expression (int2.cf, int3.cf)
5.5.1 Moving Averages
The manual’s ds3.cf smooths Y = 1, 3, 7, 6, 6 three
times with the weights gss = 0.5, 0.25: with the copy-on
end-value rule, with opt = 2 (the end values repeated), and
with opt = 2 applied twice (the repeat factor
r=2). The three output files are the manual’s Box 3.
Box 62sma: the three smoothings of ds3.cf
y <- c(1, 3, 7, 6, 6)
tda_sma(y, weights = c(0.5, 0.25))$table sel index value smoothed
1 0 1 1 1.00
2 0 2 3 3.50
3 0 3 7 5.75
4 0 4 6 6.25
5 0 5 6 6.00tda_sma(y, weights = c(0.5, 0.25), options = list(opt = 2))$table sel index value smoothed
1 0 1 1 1.50
2 0 2 3 3.50
3 0 3 7 5.75
4 0 4 6 6.25
5 0 5 6 6.00tda_sma(y, weights = c(0.5, 0.25), options = list(opt = 2, r = 2))$table sel index value smoothed
1 0 1 1 2.0000
2 0 2 3 3.5625
3 0 3 7 5.3125
4 0 4 6 6.0625
5 0 5 6 6.06255.5.2 Running Median Smoothers
A running median is resistant to the single wild value a moving average would spread over its whole window.
The manual’s ds4.cf draws 20 uniform numbers with TDA’s
generator and smooths them three ways with the smd operator
inside nvar – 3R, 4253Ht and
3RSSHt – then plots the three; tda_derive() is
the operator form, tda_smd() the command.
Box 63smd: the three smoothers of ds4.cf
gen <- tda_rng()
ds4 <- data.frame(C = 1:20, A = sapply(1:20, function(i) gen$rd()))
ds4 <- tda_derive(ds4, M1 = "smd[sm=[3R]](A)", M2 = "smd[sm=[4253Ht]](A)",
M3 = "smd[sm=[3RSSHt]](A)")
head(ds4) C A M1 M2 M3
1 1 0.02910383 0.02910383 0.02910383 0.02910383
2 2 0.94947018 0.09430404 0.28925601 0.11060409
3 3 0.09430404 0.70012522 0.53366834 0.25730455
4 4 0.70012523 0.70012522 0.62828529 0.53050137
5 5 0.89135106 0.70012522 0.57828712 0.70215678
6 6 0.47205962 0.47205961 0.46866766 0.51609015Box 64smd: ds4.cf, drawn
p <- tda_ps(ds4, width = 90, height = 50, xlim = c(0, 21), ylim = c(0, 1))
p <- tda_pl(p, "plxa", sc = 1)
p <- tda_pl(p, "plya", sc = 1, ic = 10)
p <- tda_pl_points(p, "C", "M1", symbol = 5, size = 1) # ds4.cf marks M1
p <- tda_pl_lines(p, "C", "M1")
p <- tda_pl_lines(p, "C", "M2", lty = 5)
plot(tda_pl_lines(p, "C", "M3", lty = 8))
5.5.3 Smoothing Splines
sig= is the smoothing parameter and deg=
the degree; the derivatives come back alongside the fit.
The manual’s ds5.cf is a sine curve with uniform noise,
100 points, smoothed with sig = 1, and plotted against the
curve.
Box 65spl: ds5.cf
gen <- tda_rng()
ds5 <- data.frame(X = (1:100) / 10)
ds5$Z <- sin(ds5$X)
ds5$Y <- ds5$Z + sapply(1:100, function(i) gen$rd(a = -0.2, b = 0.2))
sp5 <- tda_spl(ds5$X, ds5$Y, sig = 1)
head(sp5$table) index x y fitted d1 d2
1 1 0.1 -0.08852505 0.01562586 1.5306227 -1.5167364
2 2 0.2 0.37845740 0.16110445 1.3789490 -1.5167364
3 3 0.3 0.13324182 0.29141567 1.2272754 -1.5167364
4 4 0.4 0.46946844 0.40655953 1.0756018 -1.5167364
5 5 0.5 0.63596596 0.50653602 0.9239281 -1.5167364
6 6 0.6 0.55346632 0.59134515 0.7722545 -0.2058181ds5.cf then draws the data, the true sine and the
smoother’s fitted values.
Box 66spl: ds5.cf, drawn
p <- tda_ps(ds5, width = 90, height = 50, xlim = c(0, 10), ylim = c(-1.5, 1.5))
p <- tda_pl(p, "plxa", sc = 1)
p <- tda_pl(p, "plya", sc = 0.5, ic = 5)
p <- tda_pl_points(p, "X", "Y", symbol = 5, size = 0.5)
p <- tda_pl_lines(p, "X", "Z", lty = 5) # the sine the noise hides
plot(tda_pl_lines(p, x = sp5$table$x, y = sp5$table$fitted))
5.6.1 The fmin Command
The manual’s fmin1.cf minimises (x - 5)^2 + (y + 6)^2
from the starting values 0, 0; its box shows convergence in two
iterations to x = 5, y = -6.
Box 67fmin: fmin1.cf
tda_minimize("(x - 5)^2 + (y + 6)^2")Call: tda_minimize("(x - 5)^2 + (y + 6)^2")
Value: 0
Idx Parameter Value Error Value_E Signif
1 x 5 NA NA NA
2 y -6 NA NA NAfmin2.cf minimises a function of the data, the sum of (x
- A)^2 over A = 1, …, 101 – fmin with v = A.
With data, fmin is tda_freg()’s nonlinear
least squares; the minimum is at x = 51.
Box 68fmin with data: fmin2.cf
tda_freg({ fn = (x - A)^2 }, data = data.frame(A = 1:101))Call: tda_freg(definitions = {
fn = (x - A)^2
}, data = data.frame(A = 1:101))
Cases: 101
Converged in 2 iterations
Idx Parameter Coeff Error C/Error Signif
1 x 51.0000 2.9300 17.4060 1.00006 Statistical Procedures
6.2.1 Mean and Range of Variables
tda_dstat() is the dstat command: mean,
standard deviation and range for every variable it is given. The data
files of chapter 6 are read straight from the package.
Box 69dstat on ds1.dat
ds1 <- read.table(ex("ds1.dat"), col.names = c("X1", "X2", "X3", "X4"))
tda_dstat(ds1)Call: tda_dstat(ds1)
Cases: 4
Variable Minimum Maximum Mean Std.Dev. Sum
X1 1e+00 4.0 2.500000 1.29099445 10.0000
X2 1e+00 7.0 4.000000 3.46410162 16.0000
X3 -1e+00 2.0 -0.250000 1.50000000 -1.0000
X4 1e-04 0.1 0.027775 0.04835703 0.11116.2.2 Empirical Distribution Functions
tda_gdf() estimates the distribution function, and
what= asks for the survivor function or the expected values
instead. The manual’s gdf1.dat holds three small examples
told apart by its first column (SEL, TDA’s
tsel): exact observations (Box 3), right-censored ones with
cen= (Boxes 4 and 5), and a two-dimensional one with
grp=.
Box 70gdf: exact data (the manual’s Box 3)
gdf1 <- read.table(ex("gdf1.dat"))
names(gdf1) <- c("SEL", "ID", "L1", "YL", "DELTA")
ex0 <- gdf1[gdf1$SEL == 0, ]
tda_gdf(~YL, data = ex0)$table dimension value distribution_function
1 1 -1 0.1666667
2 1 1 0.3333333
3 1 5 0.5000000
4 1 7 0.8333333
5 1 9 1.0000000tda_gdf(~YL, data = ex0, what = "survivor")$table dimension value survivor_function
1 1 -1 8.333333e-01
2 1 1 6.666667e-01
3 1 5 5.000000e-01
4 1 7 1.666667e-01
5 1 9 1.110223e-16tda_gdf(~YL, data = ex0, what = "expected_values")$table dimension y exact expected
1 1 1 1 1
2 1 7 1 7
3 1 -1 1 -1
4 1 5 1 5
5 1 7 1 7
6 1 9 1 9TDA’s cen= variable, DELTA in the manual,
is 1 for an exact observation; censor= here is the other
way round, 1 (or TRUE) for a censored one, so the manual’s
indicator is passed negated.
Box 71gdf: right censored data (Boxes 4 and 5)
ex1 <- gdf1[gdf1$SEL == 1, ]
tda_gdf(~YL, data = ex1, censor = ex1$DELTA == 0)$table dimension value distribution_function
1 1 1 0.1666667
2 1 3 0.3750000
3 1 4 0.6875000
4 1 5 1.0000000tda_gdf(~YL, data = ex1, censor = ex1$DELTA == 0, what = "expected_values")$table dimension y exact expected
1 1 3 1 3.000
2 1 3 0 4.500
3 1 2 0 4.125
4 1 1 1 1.000
5 1 4 1 4.000
6 1 5 1 5.000The third block of gdf1.dat is the two-dimensional case:
four units, each observed in two dimensions. id= and
dimension= are TDA’s grp=, and
method= its opt= – "joint1" for
the joint distribution over the observed points.
Box 72gdf: the joint distribution (Box 6)
ex2 <- gdf1[gdf1$SEL == 2, ]
tda_gdf(~YL, data = ex2, id = ex2$ID, dimension = ex2$L1,
method = "joint1")$table pattern margin1 margin2 value1 value2 distribution_function
1 1 1 2 -1 1.0 0.25
2 1 1 2 1 2.0 0.50
3 1 1 2 2 1.0 0.50
4 1 1 2 3 1.5 0.75tda_gdf(~YL, data = ex2, id = ex2$ID, dimension = ex2$L1,
method = "joint1", what = "survivor")$table pattern margin1 margin2 value1 value2 survivor_function
1 1 1 2 3 1.5 0.00
2 1 1 2 2 1.0 0.25
3 1 1 2 1 2.0 0.00
4 1 1 2 -1 1.0 0.50With censoring the same estimator needs a grid: n_boxes=
is TDA’s n=, the number of boxes per dimension, and
sc= the scaling. The data are Pruitt’s, the manual’s Box
8.
Box 73gdf: joint distribution, censored (Boxes 9 and 10)
pruitt <- read.table(ex("gdf2.dat"))
names(pruitt) <- c("ID", "L1", "Y", "CEN")
tda_gdf(~Y, data = pruitt, censor = pruitt$CEN == 0, id = pruitt$ID,
dimension = pruitt$L1, method = "joint1",
n_boxes = 64, options = list(sc = 0.5))$table pattern margin1 margin2 value1 value2 distribution_function
1 1 1 2 1 6 4.333310e-07
2 1 1 2 2 4 3.960414e-05
3 1 1 2 3 5 7.920829e-05
4 1 1 2 4 3 1.250000e-01
5 1 1 2 5 2 5.872780e-08
6 1 1 2 6 7 6.289940e-01
7 1 1 2 7 1 3.830525e-10
8 1 1 2 8 8 1.000000e+00tda_gdf(~Y, data = pruitt, censor = pruitt$CEN == 0, id = pruitt$ID,
dimension = pruitt$L1, method = "joint1", n_boxes = 64,
options = list(sc = 0.5), what = "expected_values")$table pattern margin1 margin2 observed1 observed2 exact1 exact2 expected1 expected2
1 1 1 2 1 6 0 1 5.619719 6.000000
2 1 1 2 2 4 0 1 5.583929 4.000000
3 1 1 2 3 5 0 0 6.610054 6.984936
4 1 1 2 4 3 1 1 4.000000 3.000000
5 1 1 2 5 2 1 0 5.000000 5.156116
6 1 1 2 6 7 1 1 6.000000 7.000000
7 1 1 2 7 1 0 0 7.849248 7.154006
8 1 1 2 8 8 1 1 8.000000 8.000000The manual’s Box 12 sets the two joint methods against each other on
a hundred generated units. tda_rng() is TDA’s generator, so
the data come out the same here as in gdf3.cf.
Both methods reproduce the box: 37.3177, 38.0035 and 46.2348 for method 2, and 37.6603, 37.6603 and 46.2104 for method 1.
A few of the later rows differ in the last digit or two – 103.47752
here against the manual’s 103.4777. gdf3.cf writes its
generated data to a file and gdf4.cf reads it back at
[10.6], so its input is rounded to six decimals; this
passes R’s doubles straight to TDA. The deviance in 6.15.2.1 differs for
the same reason.
Box 74gdf: the two joint methods (Box 12)
gen <- tda_rng()
X <- 1:100
Y <- X + sapply(X, function(i) gen$rd(a = -10, b = 10))
RD1 <- sapply(X, function(i) gen$rd())
RD2 <- sapply(X, function(i) gen$rd(a = 10, b = 90))
IS <- RD1 <= 0.2 & Y > RD2
two <- rbind(data.frame(ID = X, L1 = 1L, Y1 = as.numeric(X), CEN = 1L),
data.frame(ID = X, L1 = 2L, Y1 = ifelse(IS, RD2, Y),
CEN = as.integer(!IS)))
ev <- function(m)
tda_gdf(~Y1, data = two, censor = two$CEN == 0, id = two$ID,
dimension = two$L1, method = m, what = "expected_values")$table
j1 <- ev("joint1")
j2 <- ev("joint2")
# the box lists every censored unit, the thirteen whose second
# dimension had to be estimated
i <- which(j1$exact2 == 0)
data.frame(j1[i, c("observed1", "observed2", "exact1", "exact2")],
joint1 = j1$expected2[i], joint2 = j2$expected2[i],
row.names = NULL) observed1 observed2 exact1 exact2 joint1 joint2
1 34 27.34643 1 0 37.66034 37.31767
2 36 30.39963 1 0 37.66034 38.00346
3 43 15.35934 1 0 46.21040 46.23480
4 45 43.53655 1 0 48.92635 48.47346
5 49 50.67826 1 0 54.37489 51.91668
6 68 50.51117 1 0 61.43484 71.75636
7 69 47.40821 1 0 61.43484 71.75636
8 73 49.90086 1 0 74.52141 73.02770
9 75 65.25909 1 0 74.52141 73.79130
10 79 38.74952 1 0 74.52141 79.16228
11 84 53.56255 1 0 82.37462 83.42253
12 85 62.95928 1 0 82.37462 85.37421
13 93 33.88362 1 0 99.94734 103.47752gdf3.cf does not run to completion through the real TDA
binary, so it has no box here.
6.2.3 Quantiles
Box 75quant
tda_quant(cars)Call: tda_quant(cars)
Cases: 50
p10 p20 p25 p30 p40 p50 p60 p70 p75 p80 p90
speed 8.1 11.0 12.0 12.3 14 15 17 18.7 19.25 20.0 23.9
dist 14.2 20.4 25.5 26.0 32 36 46 54.0 57.00 65.6 83.66.2.4 Frequency Tables
tda_freq() is the one-way table,
tda_freq2() the two-way one.
Box 76freq and freq2
tda_freq(ds1[c("X1", "X2", "X3")])Call: tda_freq(ds1[c("X1", "X2", "X3")])
Cases: 4
index X1 X2 X3 count percent cum_count cum_percent
1 1 1 -1 1 25 1 25
2 2 7 -1 1 25 2 50
3 3 1 -1 1 25 3 75
4 4 7 2 1 25 4 100tda_freq2(ds1$X2, ds1$X1)Call: tda_freq2(ds1$X2, ds1$X1)
Frequency|
Percent |
Row Pct |
Col Pct | 1 | 2 | 3 | 4 | Total
----------------------------------------------------------
1 | 1.00 | 0.00 | 1.00 | 0.00 | 2.00
| 25.00 | 0.00 | 25.00 | 0.00 | 50.00
| 50.00 | 0.00 | 50.00 | 0.00 |
| 100.00 | 0.00 | 100.00 | 0.00 |
----------------------------------------------------------
7 | 0.00 | 1.00 | 0.00 | 1.00 | 2.00
| 0.00 | 25.00 | 0.00 | 25.00 | 50.00
| 0.00 | 50.00 | 0.00 | 50.00 |
| 0.00 | 100.00 | 0.00 | 100.00 |
----------------------------------------------------------
Total 1.00 1.00 1.00 1.00 4.00
25.00 25.00 25.00 25.00 100.006.2.5 Aggregated Tables
tda_atab() groups a variable into classes and aggregates
a second one over them.
Box 77atab
tda_atab(ds1[c("X3", "X4")], breaks = seq(-10, 10, 0.5))Call: tda_atab(ds1[c("X3", "X4")], breaks = seq(-10, 10, 0.5))
Cases: 4
lower upper count weighted mean_X3 mean_X4
-1.0 -0.5 3 3 -1 0.0370
-0.5 0.0 0 0 0 0.0000
0.0 0.5 0 0 0 0.0000
0.5 1.0 0 0 0 0.0000
1.0 1.5 0 0 0 0.0000
1.5 2.0 0 0 0 0.0000
2.0 2.5 1 1 2 0.00016.2.6 Covariance and Correlation
Box 78corr
tda_corr(ds1)Call: tda_corr(ds1)
Cases: 4
X1 X2 X3 X4
X1 1.0000000 0.4472136 0.7745967 -0.8241407
X2 0.4472136 1.0000000 0.5773503 -0.5426423
X3 0.7745967 0.5773503 1.0000000 -0.3815371
X4 -0.8241407 -0.5426423 -0.3815371 1.00000006.2.7 Contingency Measures
The same two-way table as in 6.2.4, with
contingency = TRUE adding the measures of association.
Box 79freq2 with contingency measures
tda_freq2(ds1$X2, ds1$X1, contingency = TRUE)Call: tda_freq2(ds1$X2, ds1$X1, contingency = TRUE)
Frequency|
Percent |
Row Pct |
Col Pct | 1 | 2 | 3 | 4 | Total
----------------------------------------------------------
1 | 1.00 | 0.00 | 1.00 | 0.00 | 2.00
| 25.00 | 0.00 | 25.00 | 0.00 | 50.00
| 50.00 | 0.00 | 50.00 | 0.00 |
| 100.00 | 0.00 | 100.00 | 0.00 |
----------------------------------------------------------
7 | 0.00 | 1.00 | 0.00 | 1.00 | 2.00
| 0.00 | 25.00 | 0.00 | 25.00 | 50.00
| 0.00 | 50.00 | 0.00 | 50.00 |
| 0.00 | 100.00 | 0.00 | 100.00 |
----------------------------------------------------------
Total 1.00 1.00 1.00 1.00 4.00
25.00 25.00 25.00 25.00 100.00
----------------------------------------------------------------------------
Contingency measures for table: X (Vds1_X2) x Y (Vds1_X1)
Number of cells with less than 5 elements: 8 (100.0 %)
Cells with expected frequency less than 5: 8 (100.0 %)
Degrees of freedom: 3
Chi-square (Pearson) 4.0000
Prob: 0.2615
Chi-square (likelihood ratio) 5.5452
Prob: 0.1360
Phi 1.0000
Cramer's V 1.0000
Contingency coefficient 0.7071
Lambda (X dependent) 1.0000
Lambda (Y dependent) 0.3333
Lambda (symmetric) 0.6000
Goodman/Kruskal Tau (X dependent) 1.0000
Goodman/Kruskal Tau (Y dependent) 0.3333
Uncertainty coeff (X dependent) 1.0000
Uncertainty coeff (Y dependent) 0.5000
Kendall's tau-b 0.4082
Kendall's tau-c 0.5000
Gamma 0.5000
Somers' D (X dependent) 0.3333
Somers' D (Y dependent) 0.50006.2.8 Scatterplots
The points come from TDA’s random number generator,
tda_rng(), so that the picture is the manual’s and not one
of R’s.
Box 80Points from TDA’s generator
gen <- tda_rng()
n <- 200
X <- Y <- numeric(n)
for (i in seq_len(n)) {
X[i] <- gen$rd(a = 0, b = 3)
Y[i] <- sin(X[i]) + gen$rd()
}
df <- data.frame(x = X, y = Y)Box 81A scatterplot
p <- tda_ps(data = df, width = 90, height = 50, xlim = c(0, 3), ylim = c(-1, 2.5))
p <- tda_pl_scatter(p, "x", "y", symbol = 5, size = 1)
p <- tda_pl_axes(p, sc = c(1), ic = c(10, 0))
plot(p)
Example 1 A sunflower plot bins the same points and draws a petal per point in the bin, which is what to reach for when the points overlap.
Box 82A sunflower plot
p <- tda_ps(data = df, width = 90, height = 50, xlim = c(0, 3), ylim = c(-1, 2.5))
p <- tda_pl_scatter(p, "x", "y", type = "sunflower", grid = c(15, 10), size = 3)
p <- tda_pl_axes(p, sc = c(1), ic = c(10, 1))
plot(p)
Example 2 type = "lowess" adds the
smooth.
Box 83A scatterplot with a lowess smooth
6.2.9 Histogram Plots
Box 84A histogram (dh1.cf)
dh1 <- read.table(ex("dh1.dat"))
p <- tda_ps(data = dh1, width = 90, height = 50, xlim = c(0, 8), ylim = c(0, 0.4))
p <- tda_pl(p, "plxa", sc = 1)
p <- tda_pl(p, "plya", sc = 0.1)
p <- tda_pl_histogram(p, x = "V1", breaks = c(0, 2, 4, 6, 8))
plot(p)
6.2.10 Density Plots
The same data smoothed with a triangular kernel.
Box 85A kernel density estimate (dh2.cf)
6.4.2 Inequality Measures
Box 86ineq
tda_ineq(ds1)Call: tda_ineq(ds1)
Cases: 4
index cases minimum maximum mean sd vcoeff gini
X1 4 1e+00 4.0 2.500000 1.29099445 0.5163978 0.3333333
X2 4 1e+00 7.0 4.000000 3.46410162 0.8660254 0.5000000
X3 1 2e+00 2.0 2.000000 0.00000000 0.0000000 0.0000000
X4 4 1e-04 0.1 0.027775 0.04835703 1.7410272 0.92619266.4.3 Segregation Indices
Box 87segr
ds2 <- read.table(ex("ds2.dat"))
names(ds2) <- c("G", paste0("X", 1:4))
tda_segr(ds2, group = "G")Call: tda_segr(data = ds2, group = "G")
Cases:
variable classes cases group0 group1 d_index v_ratio gini
X1 2 7 3 4 1.0000000 1.0000000 1.0000000
X2 7 7 3 4 1.0000000 1.0000000 1.0000000
X3 3 6 3 3 0.0000000 0.0000000 0.0000000
X4 2 7 3 4 0.4166667 0.1736111 0.41666676.5.1 Life Tables
tda_ltb() is the ltb command, and
tp= gives the interval boundaries.
Box 88A life table, 30-month intervals
lt <- tda_ltb(Surv(TFP, DES) ~ 1, data = d, tp = seq(0, 500, 30))
lt$summary sn origin cases weighted median
1 1 0 600 600 47.27858lt$table start midpoint entering censored exposed events prob
1 0 15 600 28 586.0 223 0.38054608
2 30 45 349 23 337.5 113 0.33481481
3 60 75 213 15 205.5 51 0.24817518
4 90 105 147 16 139.0 25 0.17985612
5 120 135 106 15 98.5 24 0.24365482
6 150 165 67 5 64.5 9 0.13953488
7 180 195 53 9 48.5 4 0.08247423
8 210 225 40 5 37.5 3 0.08000000
9 240 255 32 5 29.5 0 0.00000000
10 270 285 27 7 23.5 2 0.08510638
11 300 315 18 5 15.5 2 0.12903226
12 330 345 11 1 10.5 2 0.19047619
13 360 375 8 3 6.5 0 0.00000000
14 390 405 5 4 3.0 0 0.00000000
15 420 435 1 1 0.5 0 0.00000000# ltb.1 holds a second block: the estimates themselves, with their
# standard errors
lt$survivor start midpoint survivor survivor_se density density_se rate rate_se
1 0 15 1.00000000 0.00000000 0.0126848692 0.0006685564 0.015665613 0.001019673
2 30 45 0.61945392 0.02005669 0.0069134117 0.0005757229 0.013404508 0.001235237
3 60 75 0.41205157 0.02076559 0.0034086992 0.0004481030 0.009444444 0.001309149
4 90 105 0.30979060 0.01994731 0.0018572578 0.0003570178 0.006587615 0.001311075
5 120 135 0.25407287 0.01922194 0.0020635360 0.0003982041 0.009248555 0.001869599
6 150 165 0.19216679 0.01822472 0.0008937990 0.0002890738 0.005000000 0.001661973
7 180 195 0.16535282 0.01773858 0.0004545782 0.0002231094 0.002867384 0.001432365
8 210 225 0.15171547 0.01753725 0.0004045746 0.0002288721 0.002777778 0.001602358
9 240 255 0.13957823 0.01747828 0.0000000000 NA 0.000000000 NA
10 270 285 0.13957823 0.01747828 0.0003959666 0.0002723627 0.002962963 0.002093061
11 300 315 0.12769923 0.01789568 0.0005492440 0.0003705354 0.004597701 0.003243325
12 330 345 0.11122191 0.01900462 0.0007061709 0.0004651940 0.007017544 0.004934585
13 360 375 0.09003679 0.02045360 0.0000000000 NA 0.000000000 NA
14 390 405 0.09003679 0.02045360 0.0000000000 NA 0.000000000 NA
15 420 435 0.09003679 0.02045360 0.0000000000 NA 0.000000000 NAtda_survivor() reads the estimate back as a data frame
carrying the survivor function, the density and the rate, which is what
the plotting commands take.
Box 89The estimate as a data frame (ple1.cf)
tda_survivor(lt) group time survivor std.err density rate lower upper
1 1 0 1.00000000 0.00000000 0.0126848692 0.015665613 1.00000000 1.0000000
2 1 30 0.61945392 0.02005669 0.0069134117 0.013404508 0.58014353 0.6587643
3 1 60 0.41205157 0.02076559 0.0034086992 0.009444444 0.37135177 0.4527514
4 1 90 0.30979060 0.01994731 0.0018572578 0.006587615 0.27069459 0.3488866
5 1 120 0.25407287 0.01922194 0.0020635360 0.009248555 0.21639855 0.2917472
6 1 150 0.19216679 0.01822472 0.0008937990 0.005000000 0.15644700 0.2278866
7 1 180 0.16535282 0.01773858 0.0004545782 0.002867384 0.13058584 0.2001198
8 1 210 0.15171547 0.01753725 0.0004045746 0.002777778 0.11734310 0.1860878
9 1 240 0.13957823 0.01747828 0.0000000000 0.000000000 0.10532142 0.1738350
10 1 270 0.13957823 0.01747828 0.0003959666 0.002962963 0.10532142 0.1738350
11 1 300 0.12769923 0.01789568 0.0005492440 0.004597701 0.09262435 0.1627741
12 1 330 0.11122191 0.01900462 0.0007061709 0.007017544 0.07397354 0.1484703
13 1 360 0.09003679 0.02045360 0.0000000000 0.000000000 0.04994847 0.1301251
14 1 390 0.09003679 0.02045360 0.0000000000 0.000000000 0.04994847 0.1301251
15 1 420 0.09003679 0.02045360 0.0000000000 0.000000000 0.04994847 0.13012516.5.2 Kaplan-Meier Procedure
tda_ple() is the product-limit estimator,
tda_km() an alias for it. Case weights are TDA’s
cwt command, and here a column name.
Box 90Product-limit estimation with case weights
rrdat2 <- read.table(ex("rrdat.2"))
names(rrdat2) <- c("DUR", "CEN", "WT", "GRP")
fit <- tda_ple(Surv(DUR, CEN) ~ as.factor(GRP), data = rrdat2, weights = "WT")
fit$blocks$`0`
id index time events censored n.risk survivor std.err cum.rate
1 0 0 0 0 0 21 1.0000000 0.00000000 0.0000000
2 0 1 6 3 0 21 0.8571429 0.07636035 0.1541507
3 0 2 7 1 1 17 0.8067227 0.08693529 0.2147753
4 0 3 10 1 1 15 0.7529412 0.09634965 0.2837682
5 0 4 13 1 2 12 0.6901961 0.10681471 0.3707796
6 0 5 16 1 0 11 0.6274510 0.11405387 0.4660897
7 0 6 22 1 3 7 0.5378151 0.12823375 0.6202404
8 0 7 23 1 0 6 0.4481793 0.13459146 0.8025620
9 0 8 35 0 5 NA NA NA NA
$`1`
id index time events censored n.risk survivor std.err cum.rate
1 1 0 0 0 0 21 1.00000000 0.00000000 0.0000000
2 1 1 1 2 0 21 0.90476190 0.06405645 0.1000835
3 1 2 2 2 0 19 0.80952381 0.08568909 0.2113091
4 1 3 3 1 0 17 0.76190476 0.09294286 0.2719337
5 1 4 4 2 0 16 0.66666667 0.10286890 0.4054651
6 1 5 5 2 0 14 0.57142857 0.10798985 0.5596158
7 1 6 8 4 0 12 0.38095238 0.10597117 0.9650809
8 1 7 11 2 0 8 0.28571429 0.09858079 1.2527630
9 1 8 12 2 0 6 0.19047619 0.08568909 1.6582281
10 1 9 15 1 0 4 0.14285714 0.07636035 1.9459101
11 1 10 17 1 0 3 0.09523810 0.06405645 2.3513753
12 1 11 22 1 0 2 0.04761905 0.04647143 3.0445224
13 1 12 23 1 0 1 0.00000000 NaN NaNple2.cf is the estimate on the job data with the single
destination state; the manual’s Box 5 shows the first rows of its output
file.
Box 91ple2.cf: the job data, one destination
ple2 <- tda_ple(Surv(TFP, DES) ~ 1, data = d)
# the manual prints part of ple.2: its first rows, a slice from the
# middle, and its last
b <- ple2$blocks[[1L]]
b[c(1:8, 41:46, 128:132), ] id index time events censored n.risk survivor std.err cum.rate
1 0 0 0 0 0 600 1.00000000 0.000000000 0.000000000
2 0 1 2 2 0 600 0.99666667 0.002353091 0.003338901
3 0 2 3 5 1 597 0.98831937 0.004389012 0.011749380
4 0 3 4 9 2 590 0.97324332 0.006599178 0.027121160
5 0 4 5 3 0 581 0.96821796 0.007174619 0.032298048
6 0 5 6 10 1 577 0.95143776 0.008796359 0.049781011
7 0 6 7 9 0 567 0.93633557 0.009993927 0.065781352
8 0 7 8 6 1 557 0.92624937 0.010700980 0.076611783
41 0 40 41 3 1 277 0.50584708 0.020924866 0.681520865
42 0 41 42 1 1 273 0.50399416 0.020930097 0.685190594
43 0 42 43 2 0 272 0.50028832 0.020939593 0.692570702
44 0 43 44 5 1 269 0.49098928 0.020959274 0.711332978
45 0 44 45 1 1 263 0.48912240 0.020962559 0.715142506
46 0 45 46 2 0 262 0.48538864 0.020968140 0.722805379
128 0 127 312 1 3 16 0.11980376 0.018459344 2.121900191
129 0 128 326 1 1 14 0.11124635 0.019021207 2.196008163
130 0 129 332 1 2 11 0.10113305 0.019798847 2.291318343
131 0 130 350 1 1 9 0.08989604 0.020541766 2.409101379
132 0 131 428 0 8 NA NA NA NA# the table ends with the last observation, where only censoring
# happened: TDA writes no estimate there, so those columns are NA
tail(ple2$blocks[[1]], 2) id index time events censored n.risk survivor std.err cum.rate
131 0 130 350 1 1 9 0.08989604 0.02054177 2.409101
132 0 131 428 0 8 NA NA NA NAple3.cf has the four destination states (Example 3): the
estimate is per transition, one sub-survivor function for each of the
upward, lateral and downward job changes, and $summary
holds one row for each. ple3p.cf plots the three:
tda_survivor() gives the estimated functions as a data
frame, and each transition is selected by its origin and
destination.
Box 92Sub-survivor functions, four destination states
ple <- tda_ple(Surv(TFP, DES) ~ 1, data = d4)
ple$summary transition median limit cases weighted
1 0,1 NA NA 600 600
2 0,2 122.2409 350 600 600
3 0,3 215.9553 332 600 600Box 93One survivor function per destination state
srv <- tda_survivor(ple)
p <- tda_ps(srv, width = 80, height = 40, xlim = c(0, 300), ylim = c(0, 1))
p <- tda_pl(p, "plxa", sc = 60, ic = 5)
p <- tda_pl(p, "plya", sc = 1, ic = 10)
p <- tda_pl_lines(p, "time", "survivor", by = "group", select = "0,1", lty = 1)
p <- tda_pl_lines(p, "time", "survivor", by = "group", select = "0,2", lty = 5)
p <- tda_pl_lines(p, "time", "survivor", by = "group", select = "0,3", lty = 8)
plot(p)
6.5.3 Survivor Function Quantiles
at_survival= is TDA’s qo=: survival
probabilities, in decreasing order, and the time each is reached at, one
row per transition and probability. A transition whose survivor function
never drops to a probability has no row for it, which is why the three
blocks are of different length. (quantiles= is the other
direction, qt=: time points, and the survivor function
reached at each.)
Box 94Quantiles of the survivor function
tda_ple(Surv(TFP, DES) ~ 1, data = d4,
at_survival = seq(0.9, 0.1, by = -0.1))$quantiles sn org des survivor quantile
1 1 0 1 0.9 30.03523
2 1 0 1 0.8 85.06017
3 1 0 1 0.7 233.87848
4 1 0 2 0.9 16.87488
5 1 0 2 0.8 27.30904
6 1 0 2 0.7 41.61867
7 1 0 2 0.6 73.19972
8 1 0 2 0.5 122.24094
9 1 0 2 0.4 208.56549
10 1 0 3 0.9 22.67992
11 1 0 3 0.8 47.12627
12 1 0 3 0.7 75.75391
13 1 0 3 0.6 131.53781
14 1 0 3 0.5 215.95529
15 1 0 3 0.4 321.988096.5.4 Comparing Survivor Functions
compare = TRUE runs TDA’s four tests of whether the
group survivor functions differ. The manual’s example,
ple6.cf, is Kalbfleisch and Prentice’s two-group data
(rrdat.3): 40 cases, survival time, status and group.
Box 95ple6.cf: comparing two survivor functions
rrdat3 <- read.table(ex("rrdat.3"), col.names = c("TF", "Status", "Group"))
tda_ple(Surv(TF, Status) ~ as.factor(Group), data = rrdat3,
compare = TRUE)$comparison sn org des test statistic df signif
1 1 0 1 Log-Rank (Savage) 3.122712 1 0.9227918
2 1 0 1 Wilcoxon (Breslow) 2.651042 1 0.8965180
3 1 0 1 Wilcoxon (Tarone-Ware) 2.976660 1 0.9155266
4 1 0 1 Wilcoxon (Prentice) 3.001409 1 0.9168079The same four tests on the job data by sex, whose survivor functions the next box draws:
Box 96Men and women
pl2 <- tda_ple(Surv(TFP, DES) ~ as.factor(SEX), data = d, compare = TRUE)
pl2$comparison sn org des test statistic df signif
1 1 0 1 Log-Rank (Savage) 20.603209 1 0.9999943
2 1 0 1 Wilcoxon (Breslow) 9.364723 1 0.9977880
3 1 0 1 Wilcoxon (Tarone-Ware) 14.326703 1 0.9998463
4 1 0 1 Wilcoxon (Prentice) 10.695087 1 0.9989258The bands are Greenwood’s, and band= shades them. They
widen where few are still at risk, so the manual cuts the plot at 290
months.
Box 97Survivor functions with confidence bands
sb <- tda_survivor(pl2)
sb <- sb[sb$time < 290, ]
p <- tda_ps(sb, width = 90, height = 50, xlim = c(0, 300), ylim = c(0, 1))
p <- tda_pl_lines(p, "time", "survivor", by = "group", select = "1",
band = c("upper", "lower"), lty = 1)
p <- tda_pl_lines(p, "time", "survivor", by = "group", select = "2",
band = c("upper", "lower"), lty = 5)
p <- tda_pl(p, "plxa", sc = 60, ic = 1)
p <- tda_pl(p, "plya", sc = 1, ic = 10)
p <- tda_pl_frame(p)
plot(p)
6.5.5 State Distributions
tda_state_dist() is epsdat: at each time
point, how many episodes are in each state. The manual’s
ed5.cf runs it on the small multi-state file
ed1.dat; with the origin state on the left-hand side
(Surv(ts, tf, org, des)) the counts are those of its Box 2,
with the file’s Total and Missing columns.
Box 98epsdat: state distributions (ed5.cf)
ed1 <- read.table(ex("ed1.dat"),
col.names = c("ID", "SN", "ORG", "DES", "TS", "TF", "S1", "S2"))
tda_state_dist(Surv(TS, TF, ORG, DES) ~ 1, ed1, times = seq(0, 30, 5)) time state1 state2 state3 total missing
1 0 1 1 0 2 3
2 5 1 1 0 2 3
3 10 0 2 0 2 3
4 15 0 1 1 2 3
5 20 1 0 1 2 3
6 25 1 0 1 2 3
7 30 1 0 0 1 4# Box 2's second table: the same file read as multi-episode. Naming the
# id (and the spell number) puts a case's later spells with it instead
# of counting them as missing.
tda_state_dist(Surv(TS, TF, ORG, DES) ~ 1, ed1, times = seq(0, 30, 5),
id = "ID", spell = "SN") time state1 state2 state3 total missing
1 0 1 1 0 2 0
2 5 1 1 0 2 0
3 10 0 2 0 2 0
4 15 0 1 1 2 0
5 20 1 0 1 2 0
6 25 1 0 1 2 0
7 30 1 0 0 1 16.6.1 Sequence Length and Gaps
Sequence data are one row per case, one column per position.
Box 99seqlg: length and gaps
6.6.2 General Characteristics
Box 100seqgc
tda_seqgc(seqd1, id = "ID") case slen nds nev dur1 dur3 dur7 durm nep1 nep3 nep7 nepm id
1 1 8 3 4 1 4 3 0 1 2 2 0 1
2 2 8 3 4 2 3 1 2 1 2 1 1 2
3 3 7 2 2 4 2 0 1 1 1 0 1 3
4 4 6 2 1 2 4 0 0 1 1 0 0 4
5 5 8 3 4 1 4 3 0 1 2 2 0 56.6.3 State Distributions
Box 101seqsd and seqen
tda_seqsd(seqd1[, -1]) time nst1 nst3 nst7 valid nmiss total
1 0 1 1 2 4 1 5
2 1 1 4 0 5 0 5
3 2 0 4 1 5 0 5
4 3 3 1 0 4 1 5
5 4 2 3 0 5 0 5
6 5 1 3 0 4 1 5
7 6 1 0 2 3 2 5
8 7 1 1 2 4 1 5tda_seqen(seqd1[, -1]) time n ent
1 0 4 1.0397208
2 1 5 0.5004024
3 2 5 0.5004024
4 3 4 0.5623351
5 4 5 0.6730117
6 5 4 0.5623351
7 6 3 0.6365142
8 7 4 1.03972086.6.4 Pattern Matching
Patterns are given as a list: "-" is a gap, a vector is
a run of states, and "*" matches anything.
Box 102seqpm
6.7.2.2 The seqm Command
tda_seqm() is seqm: the optimal matching
distance between every pair of sequences.
Box 103seqm (seqm1.cf)
seqmd1 <- read.table(ex("seqm.d1"))
tda_seqm(seqmd1) 1 2 3
2 2
3 2 2
4 4 2 2print= asks for the alignment itself rather than the
distance, either as the sequence of operations or as the longest common
subsequence.
Box 104The alignment behind the distance
s <- tda_seqm(seqmd1, print = "sequential")
attr(s, "pairs") i j len.i len.j dist_t1 dist_t2 dist_t3 dist_t4 dist_t5 distance
1 2 1 5 5 0 0 2 2 2 2
2 3 1 5 5 0 0 0 2 2 2
3 3 2 5 5 0 0 2 2 2 2
4 4 1 5 5 0 0 2 4 4 4
5 4 2 5 5 0 0 0 2 2 2
6 4 3 5 5 0 0 2 2 2 2attr(s, "dist_by_time")[, , "t4"] [,1] [,2] [,3] [,4]
[1,] 0 2 2 4
[2,] 2 0 2 2
[3,] 2 2 0 2
[4,] 4 2 2 0s <- tda_seqm(seqmd1, print = "lcs")
attr(s, "lcs")NULL6.7.2.3 Examples
With subcost= the substitution cost is a matrix rather
than a constant, and dp_matrix = TRUE keeps the dynamic
programming table behind each distance.
Box 105A substitution cost matrix
seqmd3 <- read.table(ex("seqm.d3"))
scost <- matrix(0, 10, 10)
for (i in 1:10) for (j in 1:10) scost[i, j] <- 0.1 * abs(i - j)
r <- tda_seqm(seqmd3, indel = 1, subcost = scost, dp_matrix = TRUE)
dp <- attr(r, "dp_matrix")[["2_1"]]
dp$seqA # the two sequences the matrix aligns[1] 1 1 2 3 4 5 6dp$seqB[1] 4 5 6 7 7 7 7 7dp$D B 4 5 6 7 7 7 7 7
A 0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0
1 1 0.3 1.3 2.3 3.3 4.3 5.3 6.3 7.3
1 2 1.3 0.7 1.7 2.7 3.7 4.7 5.7 6.7
2 3 2.2 1.6 1.1 2.1 3.1 4.1 5.1 6.1
3 4 3.1 2.4 1.9 1.5 2.5 3.5 4.5 5.5
4 5 4.0 3.2 2.6 2.2 1.8 2.8 3.8 4.8
5 6 5.0 4.0 3.3 2.8 2.4 2.0 3.0 4.0
6 7 6.0 5.0 4.0 3.4 2.9 2.5 2.1 3.1s= asks seqm to print, for every pair, more than the one
distance. print = "sequential" gives the distance after
each time point (seqm1b.cf); print = "lcs"
gives the length of the longest common subsequence and the positions it
occupies (seqm1c.cf).
Box 106seqm’s s= option (seqm1b.cf, seqm1c.cf)
seqmd1 <- read.table(ex("seqm.d1"), col.names = paste0("Y", 1:5))
attr(tda_seqm(seqmd1, print = "sequential"), "pairs") i j len.i len.j dist_t1 dist_t2 dist_t3 dist_t4 dist_t5 distance
1 2 1 5 5 0 0 2 2 2 2
2 3 1 5 5 0 0 0 2 2 2
3 3 2 5 5 0 0 2 2 2 2
4 4 1 5 5 0 0 2 4 4 4
5 4 2 5 5 0 0 0 2 2 2
6 4 3 5 5 0 0 2 2 2 2lcs <- attr(tda_seqm(seqmd1, print = "lcs"), "pairs")
lcs i j len.i len.j distance lcs_length lcs_t1 lcs_t2 lcs_t3 lcs_t4 lcs_t5
1 2 1 5 5 2 4 0 0 0 0 -1
2 3 1 5 5 2 4 0 0 0 0 -1
3 3 2 5 5 2 4 0 0 1 0 -1
4 4 1 5 5 4 3 0 0 0 -1 -1
5 4 2 5 5 2 4 0 0 1 0 -1
6 4 3 5 5 2 4 0 0 1 0 -16.7.2.4 Consecutive Indel Operations
A second indel cost charges consecutive insertions or
deletions differently from the first.
Box 107indel costs (seqm4.cf)
6.7.2.5 Data-based Substitution Costs
subcost = 2 is TDA’s scost = 2:
substitution costs derived from the data, 2 - p(a,b) - p(b,a) with p the
observed transition probabilities between the two states, the manual’s
seqm5.cf. The distances are those of its output file
seqm5.d.
Box 108seqm5.cf: substitution costs based on the data
tda_seqm(seqmd4, subcost = 2) 1 2 3 4
2 1.333333
3 2.666667 1.333333
4 4.000000 2.666667 1.333333
5 5.333333 4.000000 2.666667 1.3333336.9.1 The lsreg Command
tda_lsreg() is least squares regression, and
summary() prints it the way R prints a regression.
Box 109lsreg
lsreg1 <- read.table(ex("lsreg1.dat"))
names(lsreg1) <- c("Height", "Weight")
summary(tda_lsreg(Weight ~ Height, data = lsreg1))Call: tda_lsreg(formula = Weight ~ Height, data = lsreg1)
Cases: 19
Estimate Std. Error C/Error Pr(>|t|)
Intercept -143.02692 32.27459 -4.4316 0.0003656 ***
Height 3.89903 0.51609 7.5549 7.887e-07 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 11.23 on 17 degrees of freedom
Sum of squared residuals: 2142.49
Multiple R-squared: 0.7705, Adjusted R-squared: 0.757
F-statistic: 57.0763, p-value: 7.887e-076.9.1.2 Parameter Constraints
Constraints are written the way TDA writes them, as strings over the
coefficient names b1, b2 and so on.
Box 110Equality and inequality constraints
lsreg2 <- read.table(ex("lsreg2.dat"))
names(lsreg2) <- c("X1", "X2", "Y")
tda_lsreg(Y ~ 0 + X1 + X2, data = lsreg2,
equality = "b1 + b2 = 6",
inequality = c("b1 = 2", "b2 = 2"))Call: tda_lsreg(formula = Y ~ 0 + X1 + X2, data = lsreg2, equality = "b1 + b2 = 6",
inequality = c("b1 = 2", "b2 = 2"))
Cases: 3
Idx Wave Variable Coeff Error C/Error Signif
1 1.0000 X1 3.0000 NA NA NA
2 1.0000 X2 3.0000 NA NA NA6.9.1.3 Dummy Variables
dgroup= is TDA’s group-of-dummies parameter: one
regression per group, reported in one table.
Box 111A regression per region
lsreg3 <- read.table(ex("lsreg3.dat"))
names(lsreg3) <- c("X1", "G", "Y", "NE", "NC", "SO", "WE")
lsreg3$Y2 <- lsreg3$Y * lsreg3$Y
reg <- tda_lsreg(G ~ Y + Y2, data = lsreg3,
dgroup = c("NE", "NC", "SO", "WE"))
summary(reg)Call: tda_lsreg(formula = G ~ Y + Y2, data = lsreg3, dgroup = c("NE",
"NC", "SO", "WE"))
Cases: 48
Estimate Std. Error C/Error Pr(>|t|)
Intercept 1.0452e+00 1.9787e-01 5.2823 4.238e-06 ***
Y 7.7782e-04 2.9829e-05 26.0758 < 2.2e-16 ***
Y2 -1.3325e-08 7.9539e-10 -16.7526 < 2.2e-16 ***
dgroupNC -3.9291e-02 1.3511e-01 -0.2908 0.772638
dgroupNE -7.8736e-01 1.3512e-01 -5.8272 7.039e-07 ***
dgroupSO 4.2968e-01 1.3517e-01 3.1789 0.002775 **
dgroupWE 3.9697e-01 1.3508e-01 2.9387 0.005334 **
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.5402 on 42 degrees of freedom
Sum of squared residuals: 12.2578
Multiple R-squared: 0.9739, Adjusted R-squared: 0.9708# the per-group table TDA prints for dgrp=: the cases in each and its
# weight
reg$groups variable cases weight
1 DgrpNC 12 0.25
2 DgrpNE 12 0.25
3 DgrpSO 12 0.25
4 DgrpWE 12 0.25# and the fit statistics the manual's box reports alongside them
unlist(reg$stats) df rss sigma2 r2 adj rank rnorm
42.0000000 12.2578038 0.2918525 0.9738779 0.9707681 6.0000000 3.5011147 6.10.1 L1-Norm Regression
Box 112l1reg
l1reg4 <- read.table(ex("l1reg4.dat"))
names(l1reg4) <- c("Y", "X1", "X2")
tda_l1reg(Y ~ X1 + X2, data = l1reg4)Call: tda_l1reg(formula = Y ~ X1 + X2, data = l1reg4)
Cases: 6
Idx Wave Variable Coeff Error C/Error Signif
1 NA Intercept 32.7143 16.8621 1.9401 0.8523
2 1.0000 X1 1.5952 0.9462 1.6860 0.8096
3 1.0000 X2 -0.0952 0.5537 -0.1720 0.12566.10.2 Nonparametric Regression
tda_npreg() smooths with a kernel, against the curve the
data were generated from.
Box 113A curve that is hard to recover, plus noise
gen <- tda_rng()
n <- 256
X <- E <- numeric(n)
for (i in seq_len(n)) {
X[i] <- gen$rd()
E[i] <- 0.32 * gen$rdn()
}
Z <- sin(2 * pi * X * X * X)
npreg <- data.frame(X = X, Y = Z * Z * Z + E)Box 114npreg: the kernel estimate
np <- tda_npreg(Y ~ X, data = npreg, method = "mean", kernel = "quartic",
bandwidth = 0.1, x = seq(0, 1, 0.01))
head(np$table) RECN X NX XM YM YSD
1 1 0.00 20 0.02631351 0.06019139 0.3340911
2 2 0.01 20 0.02631351 0.05075058 0.3340911
3 3 0.02 23 0.03162400 0.03975492 0.3444770
4 4 0.03 23 0.03162400 0.03203011 0.3444770
5 5 0.04 27 0.03943625 0.02852610 0.3244078
6 6 0.05 31 0.04654109 0.02926144 0.3091753Box 115npreg, against the curve it is estimating
p <- tda_ps(npreg, width = 90, height = 50, xlim = c(0, 1), ylim = c(-1.2, 1.8))
p <- tda_pl_axes(p, sc = c(0.1, 0.5), ic = 0)
p <- tda_pl_points(p, "X", "Y", symbol = 5, size = 0.5, lty = 0)
p <- tda_pl_function(p, "sin(2*pi*x*x*x)*sin(2*pi*x*x*x)*sin(2*pi*x*x*x)",
range = c(0, 1), step = 0.01, lty = "dashed")
p <- tda_pl_lines(p, x = np$table$X, y = np$table$YM, lty = "solid", lw = 1)
plot(p)
6.11.2 The fml Command
tda_fml() maximises a likelihood the user writes. The
block in braces is TDA’s function definition, with fn the
value summed over the data matrix; the names that are neither variables
nor already defined are the parameters.
The likelihood below is the exponential transition rate model of
6.17.2.1 written out by hand, and it reaches the estimates
tda_rate(model = 2) reaches.
Box 116An exponential model written as a likelihood (frml1.cf)
d$DUR <- d$TFin - d$TStart + 1L
d$W <- as.integer(d$SEX == 2)
tda_fml({
rate = exp(a0 + COHO2 * a1 + COHO3 * a2 + W * a3)
l1 = ifelse(DES, log(rate), 0)
fn = l1 - rate * DUR
}, data = d, start = c(-4, 0, 0, 0))Call: tda_fml(definitions = {
rate = exp(a0 + COHO2 * a1 + COHO3 * a2 + W * a3)
l1 = ifelse(DES, log(rate), 0)
fn = l1 - rate * DUR
}, data = d, start = c(-4, 0, 0, 0))
Cases: 600
logLik (starting values): -2578.948
logLik: -2475.438
Converged in 6 iterations
Idx Parameter Coeff Error C/Error Signif
1 a0 -5.0114 0.0843 -59.4446 1.0000
2 a1 0.5341 0.1120 4.7686 1.0000
3 a2 0.6738 0.1152 5.8472 1.0000
4 a3 0.5065 0.0942 5.3746 1.00006.12.2 Binary Logit and Probit
tda_qreg() is the qreg command, model 1 the
logit and model 2 the probit. predictions= and
standardized= ask for the two further tables.
Box 117Binary logit (qr1.cf)
qr1 <- read.table(ex("qr1.dat"))
names(qr1) <- c("Dose", "Weight", "Response")
qr1$Log10Dose <- log(qr1$Dose) / log(10)
logit <- tda_qreg(Response ~ Log10Dose, data = qr1, weights = "Weight",
predictions = TRUE, standardized = TRUE)
logitCall: tda_qreg(formula = Response ~ Log10Dose, data = qr1, standardized = TRUE,
predictions = TRUE, weights = "Weight")
Cases: 13
Model: logit (TDA 1)
logLik (starting values): -51.29289
logLik: -37.11065
Converged in 6 iterations
Idx Cat Term Variable Coeff Error C/Error Signif
1 1.0000 I Intercept -3.2246 0.8861 -3.6393 0.9997
2 1.0000 X Log10Dose 5.9702 1.4492 4.1197 1.0000logit$categories category n pct
1 0 36 48.64865
2 1 38 51.35135attr(logit$categories, "observations") 1
74 # qr1.cf fits the probit beside the logit, on the same data
tda_qreg(Response ~ Log10Dose, data = qr1, weights = "Weight",
model = "probit")Call: tda_qreg(formula = Response ~ Log10Dose, data = qr1, model = "probit",
weights = "Weight")
Cases: 13
Model: probit (TDA 2)
logLik (starting values): -51.29289
logLik: -37.28039
Converged in 5 iterations
Idx Cat Term Variable Coeff Error C/Error Signif
1 1.0000 I Intercept -1.8127 0.4493 -4.0341 0.9999
2 1.0000 X Log10Dose 3.4181 0.7455 4.5847 1.0000logit$standardized Idx Cat Term Variable Coeff Exp(C) Exp(C*SD) Std.Dev.
1 1 1 I Intercept -3.224644 0.03976993 1.00000 0.0000000
2 2 1 X Log10Dose 5.970180 391.57614388 5.12469 0.2737053logit$predictions CaseID Wave Response Log10Dose CWt PROB PROB0 PROB1
1 1 1 0 0.0000000 9 0.96175122 0.9617512 0.03824878
2 2 1 1 0.0000000 1 0.03824878 0.9617512 0.03824878
3 3 1 0 0.3010300 10 0.80650228 0.8065023 0.19349772
4 4 1 1 0.3010300 2 0.19349772 0.8065023 0.19349772
5 5 1 0 0.4771213 6 0.59294286 0.5929429 0.40705714
6 6 1 1 0.4771213 4 0.40705714 0.5929429 0.40705714
7 7 1 0 0.6020600 5 0.40859846 0.4085985 0.59140154
8 8 1 1 0.6020600 5 0.59140154 0.4085985 0.59140154
9 9 1 0 0.6989700 4 0.27922029 0.2792203 0.72077971
10 10 1 1 0.6989700 8 0.72077971 0.2792203 0.72077971
11 11 1 0 0.7781513 2 0.19449565 0.1944957 0.80550435
12 12 1 1 0.7781513 8 0.80550435 0.1944957 0.80550435
13 13 1 1 0.8450980 10 0.86065528 0.1393447 0.86065528Box 118The same data, probit
tda_qreg(Response ~ Log10Dose, data = qr1, weights = "Weight", model = 2)Call: tda_qreg(formula = Response ~ Log10Dose, data = qr1, model = 2,
weights = "Weight")
Cases: 13
Model: probit (TDA 2)
logLik (starting values): -51.29289
logLik: -37.28039
Converged in 5 iterations
Idx Cat Term Variable Coeff Error C/Error Signif
1 1.0000 I Intercept -1.8127 0.4493 -4.0341 0.9999
2 1.0000 X Log10Dose 3.4181 0.7455 4.5847 1.0000The two link functions side by side, the logistic and, through TDA’s
nd(), the standard normal distribution function.
Box 119plot-bin.cf
p <- tda_ps(xlim = c(-4, 4), ylim = c(0, 1), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(1, 1), ic = c(5, 10))
p <- tda_pl_function(p, "exp(x1)/(1+exp(x1))", range = c(-4, 4), step = 0.05)
p <- tda_pl_function(p, "nd(x1)", range = c(-4, 4), step = 0.05, lty = 6)
p <- tda_pl_function(p, "nd(x1*sqrt(3)/pi)", range = c(-4, 4), step = 0.05,
lty = 2)
plot(p)
6.12.2.1 Grouped Data
With grouped data the likelihood is binomial, and a random effect can
be integrated out with intn, TDA’s numerical integration
operator. The starting values come from the fit without the random
effect.
Box 120A grouped logit, with and without a random effect (qr5.cf)
qr5 <- read.table(ex("qr5.dat"))
names(qr5) <- c("Y", "N", "S", "X")
qr5$SX <- qr5$S * qr5$X
fml <- tda_fml({
xb = beta0 + S * beta1 + X * beta2 + SX * beta3
ee = bc(N, Y) * (exp(xb)^Y) / ((1 + exp(xb))^N)
fn = log(ee)
}, data = qr5)
fmlCall: tda_fml(definitions = {
xb = beta0 + S * beta1 + X * beta2 + SX * beta3
ee = bc(N, Y) * (exp(xb)^Y)/((1 + exp(xb))^N)
fn = log(ee)
}, data = qr5)
Cases: 21
logLik (starting values): -87.83176
logLik: -54.93702
Converged in 4 iterations
Idx Parameter Coeff Error C/Error Signif
1 beta0 -0.5582 0.1260 -4.4292 1.0000
2 beta1 0.1459 0.2232 0.6539 0.4868
3 beta2 1.3182 0.1775 7.4277 1.0000
4 beta3 -0.7781 0.3064 -2.5392 0.9889tda_fml({
xb = beta0 + S * beta1 + X * beta2 + SX * beta3
ee = intn(7, bc(N, Y) * (exp(xb + t * gam)^Y) / ((1 + exp(xb + t * gam))^N))
fn = log(ee)
}, data = qr5, start = c(coef(fml), gam = 0.1), options = list(mina = 4))Call: tda_fml(definitions = {
xb = beta0 + S * beta1 + X * beta2 + SX * beta3
ee = intn(7, bc(N, Y) * (exp(xb + t * gam)^Y)/((1 + exp(xb +
t * gam))^N))
fn = log(ee)
}, data = qr5, start = c(coef(fml), gam = 0.1), options = list(mina = 4))
Cases: 21
logLik (starting values): -54.43748
logLik: -53.75618
Converged in 17 iterations
Idx Parameter Coeff Error C/Error Signif
1 beta0 -0.5481 0.1668 -3.2861 0.9990
2 beta1 0.0965 0.2785 0.3464 0.2710
3 beta2 1.3369 0.2369 5.6442 1.0000
4 beta3 -0.8104 0.3853 -2.1034 0.9646
5 gam 0.2368 0.1110 2.1335 0.96716.12.3 Ordinal Logit and Probit
Models 3 and 4.
Box 121Ordinal logit and probit (qr2.cf)
qr2 <- read.table(ex("qr2.dat"), col.names = c("Y", "X1", "X2"))
qr_ol <- tda_qreg(Y ~ X1 + X2, data = qr2, model = 3, predictions = TRUE)
qr_olCall: tda_qreg(formula = Y ~ X1 + X2, data = qr2, model = 3, predictions = TRUE)
Cases: 20
Model: ordinal_logit (TDA 3)
logLik (starting values): -27.52453
logLik: -24.46419
Converged in 5 iterations
Idx Cat Term Variable Coeff Error C/Error Signif
1 NA I 1 Alpha 1 -1.7466 1.3020 -1.3414 0.8202
2 NA I 2 Alpha 2 -3.1531 1.4312 -2.2032 0.9724
3 NA I 3 Alpha 3 -4.2253 1.5565 -2.7146 0.9934
4 NA X X1 1.0121 0.4637 2.1826 0.9709
5 NA X X2 1.3100 0.8949 1.4639 0.8568qr_ol$categories category n pct
1 0 5 25
2 1 5 25
3 2 4 20
4 3 6 30# the manual's Box 5 is this table in full: the observed values, the
# fitted probability of the observed category, and the cumulative
# probabilities per category
qr_ol$predictions CaseID Wave Y X1 X2 PROB PROB0 PROB1 PROB2 PROB3
1 1 1 2 1.163 1 0.1495466 0.6771098 0.3393837 0.14954660 0
2 2 1 2 3.453 0 0.3250870 0.8517246 0.5845861 0.32508700 0
3 3 1 2 3.724 0 0.3878845 0.8831361 0.6492870 0.38788453 0
4 4 1 0 1.090 0 0.3444810 0.3444810 0.1140573 0.04220597 0
5 5 1 3 3.819 0 0.0000000 0.8926997 0.6708552 0.41094402 0
6 6 1 1 2.838 1 0.7367641 0.9195154 0.7367641 0.48927598 0
7 7 1 3 1.516 1 0.0000000 0.7498459 0.4234144 0.20086577 0
8 8 1 3 1.417 1 0.0000000 0.7305852 0.3991592 0.18526292 0
9 9 1 0 2.436 1 0.8838000 0.8838000 0.6507540 0.38941669 0
10 10 1 2 3.300 1 0.6046038 0.9480127 0.8170975 0.60460383 0
11 11 1 1 1.058 1 0.3159792 0.6534528 0.3159792 0.13652803 0
12 12 1 0 2.607 1 0.9004277 0.9004277 0.6889946 0.43126384 0
13 13 1 0 2.162 0 0.6086348 0.6086348 0.2758811 0.11536180 0
14 14 1 1 2.145 0 0.2724572 0.6045289 0.2724572 0.11361750 0
15 15 1 0 1.103 0 0.3474582 0.3474582 0.1153936 0.04274106 0
16 16 1 1 1.914 0 0.2286438 0.5475008 0.2286438 0.09211290 0
17 17 1 3 3.610 0 0.0000000 0.8706920 0.6225833 0.36086955 0
18 18 1 3 3.579 1 0.0000000 0.9602942 0.8555955 0.66975113 0
19 19 1 3 3.379 1 0.0000000 0.9518150 0.8287452 0.62354929 0
20 20 1 1 3.175 0 0.5150621 0.8125750 0.5150621 0.26661731 0tda_qreg(Y ~ X1 + X2, data = qr2, model = 4)Call: tda_qreg(formula = Y ~ X1 + X2, data = qr2, model = 4)
Cases: 20
Model: ordinal_probit (TDA 4)
logLik (starting values): -29.52842
logLik: -24.57412
Converged in 5 iterations
Idx Cat Term Variable Coeff Error C/Error Signif
1 NA I 1 Alpha 1 -1.0868 0.8207 -1.3243 0.8146
2 NA I 2 Alpha 2 -1.9059 0.8692 -2.1927 0.9717
3 NA I 3 Alpha 3 -2.5413 0.9193 -2.7645 0.9943
4 NA X X1 0.6122 0.2801 2.1853 0.9711
5 NA X X2 0.7563 0.5225 1.4475 0.8522The same ordinal logit written as a likelihood, which is how the
manual shows what the model is. eexp is TDA’s logistic
function.
Box 122The ordinal logit as a likelihood
tda_fml({
xb = X1 * beta1 + X2 * beta2
theta0 = 1
theta1 = eexp(alpha1 + xb)
theta2 = eexp(alpha2 + xb)
theta3 = eexp(alpha3 + xb)
ll = ifelse(Y == 0, theta0 - theta1,
ifelse(Y == 1, theta1 - theta2,
ifelse(Y == 2, theta2 - theta3, theta3)))
fn = log(ll)
}, data = qr2, start = c(-2, -3, -4, 0, 0))Call: tda_fml(definitions = {
xb = X1 * beta1 + X2 * beta2
theta0 = 1
theta1 = eexp(alpha1 + xb)
theta2 = eexp(alpha2 + xb)
theta3 = eexp(alpha3 + xb)
ll = ifelse(Y == 0, theta0 - theta1, ifelse(Y == 1, theta1 -
theta2, ifelse(Y == 2, theta2 - theta3, theta3)))
fn = log(ll)
}, data = qr2, start = c(-2, -3, -4, 0, 0))
Cases: 20
logLik (starting values): -52.01614
logLik: -24.46419
Converged in 7 iterations
Idx Parameter Coeff Error C/Error Signif
1 alpha1 -1.7466 1.3020 -1.3414 0.8202
2 alpha2 -3.1531 1.4312 -2.2032 0.9724
3 alpha3 -4.2253 1.5565 -2.7146 0.9934
4 beta1 1.0121 0.4637 2.1826 0.9709
5 beta2 1.3100 0.8949 1.4639 0.85686.12.4 Multinomial Logit
Model 5, with nq= the number of response categories.
Box 123Multinomial logit (qr3.cf)
qr3 <- read.table(ex("qr3.dat"), col.names = c("X", "Weight", "Y"))
ml <- tda_qreg(Y ~ X, data = qr3, weights = "Weight", model = 5, nq = 5)
mlCall: tda_qreg(formula = Y ~ X, data = qr3, model = 5, nq = 5, weights = "Weight")
Cases: 45
Model: multinomial_logit (TDA 5)
logLik (starting values): -935.0834
logLik: -826.7484
Converged in 7 iterations
Idx Cat Term Variable Coeff Error C/Error Signif
1 2.0000 I Intercept 7.2428 1.3578 5.3342 1.0000
2 2.0000 X X -0.5035 0.1024 -4.9187 1.0000
3 3.0000 I Intercept 8.3752 1.3438 6.2326 1.0000
4 3.0000 X X -0.5842 0.1014 -5.7582 1.0000
5 4.0000 I Intercept 11.3662 1.3897 8.1788 1.0000
6 4.0000 X X -0.8723 0.1073 -8.1304 1.0000
7 5.0000 I Intercept 11.1873 1.5042 7.4373 1.0000
8 5.0000 X X -0.9375 0.1202 -7.7979 1.0000# the distribution over the response's categories, which qreg prints
# before it estimates
ml$categories category n pct
1 1 70 12.048193
2 2 149 25.645439
3 3 175 30.120482
4 4 132 22.719449
5 5 55 9.466437attr(ml$categories, "observations") # the weighted total TDA prints 1
581 lvl() is TDA’s level operator, which takes a set of
columns as the levels of one response.
Box 124A multinomial logit over lvl() columns (qr4.cf)
qr4 <- read.table(ex("qr4.dat"), col.names = c("Z1", "Z2", "Z3", "Y"))
cml <- tda_qreg(Y ~ 0 + lvl(Z1, Z2, Z3), data = qr4, model = 5, nq = 3,
predictions = TRUE)
cmlCall: tda_qreg(formula = Y ~ 0 + lvl(Z1, Z2, Z3), data = qr4, model = 5,
nq = 3, predictions = TRUE)
Cases: 50
Model: multinomial_logit (TDA 5)
logLik (starting values): -54.93061
logLik: -33.35814
Converged in 6 iterations
Idx Cat Term Variable Coeff Error C/Error Signif
1 NA Z1 Lvl1_1 -0.3568 0.0776 -4.5966 1.0000cml$categories category n pct
1 1 14 28
2 2 29 58
3 3 7 14attr(cml$categories, "observations") 1
50 # the manual's Box 5 is this table in full
cml$predictions CaseID Wave Y Lvl1_1 Lvl1_2 Lvl1_3 PROB PROB0 PROB1 PROB2
1 1 1 2 16.481 16.196 23.890 0.50825625 0.459107699 0.50825625 0.032636051
2 2 1 2 15.123 11.373 14.182 0.61374895 0.161001851 0.61374895 0.225249200
3 3 1 2 19.469 8.822 20.819 0.96505207 0.021603374 0.96505207 0.013344553
4 4 1 2 18.847 15.649 21.280 0.68799081 0.219770542 0.68799081 0.092238651
5 5 1 2 12.578 10.671 18.335 0.63643005 0.322263785 0.63643005 0.041306161
6 6 1 1 11.513 20.582 27.838 0.95945062 0.959450618 0.03771773 0.002831652
7 7 1 1 10.651 15.537 17.418 0.79095984 0.790959836 0.13833811 0.070702054
8 8 1 1 8.359 15.675 21.050 0.92227190 0.922271898 0.06777270 0.009955405
9 9 1 1 11.679 12.668 23.104 0.58153318 0.581533182 0.40860461 0.009862205
10 10 1 2 23.237 10.356 21.346 0.97097408 0.009794046 0.97097408 0.019231879
11 11 1 3 13.236 16.019 10.087 0.69180771 0.224887888 0.08330440 0.691807713
12 12 1 3 20.052 16.861 14.168 0.66444655 0.081391947 0.25416150 0.664446554
13 13 1 2 18.917 14.764 21.564 0.76015053 0.172696842 0.76015053 0.067152626
14 14 1 2 18.200 6.868 19.095 0.97061987 0.017016157 0.97061987 0.012363971
15 15 1 1 10.777 16.554 15.938 0.77771716 0.777717162 0.09897500 0.123307841
16 16 1 2 20.003 6.377 9.314 0.73618664 0.005692268 0.73618664 0.258121097
17 17 1 2 19.768 8.523 18.960 0.95949794 0.017351589 0.95949794 0.023150471
18 18 1 2 8.151 13.845 17.643 0.11253321 0.858447817 0.11253321 0.029018969
19 19 1 1 22.173 18.045 15.535 0.06231900 0.062318999 0.27187000 0.665811004
20 20 1 2 13.134 11.067 19.108 0.65146835 0.311571688 0.65146835 0.036959964
21 21 1 1 14.051 14.247 15.764 0.40402308 0.404023080 0.37673059 0.219246332
22 22 1 3 14.685 10.811 12.361 0.31496031 0.137433204 0.54760649 0.314960309
23 23 1 1 11.666 10.758 16.445 0.38995855 0.389958552 0.53918355 0.070857901
24 24 1 3 17.211 15.201 17.059 0.25721184 0.243632153 0.49915601 0.257211837
25 25 1 1 13.930 16.227 22.024 0.66833940 0.668339401 0.29445386 0.037206734
26 26 1 2 15.237 14.345 19.984 0.53732601 0.390840266 0.53732601 0.071833721
27 27 1 1 10.840 11.071 10.188 0.31418570 0.314185698 0.28932567 0.396488636
28 28 1 2 16.841 11.224 13.417 0.62815088 0.084637718 0.62815088 0.287211398
29 29 1 3 13.913 16.991 26.618 0.00799063 0.743961963 0.24804741 0.007990630
30 30 1 2 13.089 9.822 19.162 0.74219373 0.231318722 0.74219373 0.026487545
31 31 1 3 16.626 10.725 15.285 0.14904522 0.092362412 0.75859237 0.149045217
32 32 1 2 13.477 15.509 24.421 0.32190207 0.664714178 0.32190207 0.013383753
33 33 1 2 20.851 14.557 19.800 0.79377546 0.083999927 0.79377546 0.122224608
34 34 1 2 11.365 12.673 22.212 0.38051021 0.606840956 0.38051021 0.012648835
35 35 1 2 13.296 10.076 17.810 0.72451063 0.229626567 0.72451063 0.045862804
36 36 1 1 15.417 14.103 21.050 0.36600531 0.366005308 0.58495952 0.049035176
37 37 1 2 15.938 11.180 19.851 0.81407769 0.149035656 0.81407769 0.036886649
38 38 1 2 19.034 14.125 19.764 0.76501914 0.132707436 0.76501914 0.102273425
39 39 1 1 10.466 12.481 18.540 0.64796751 0.647967508 0.31570150 0.036330990
40 40 1 3 15.799 16.979 13.074 0.61486143 0.232523970 0.15261460 0.614861430
41 41 1 2 12.713 15.105 13.629 0.19835896 0.465751592 0.19835896 0.335889446
42 42 1 2 16.908 10.958 19.713 0.85938894 0.102821215 0.85938894 0.037789847
43 43 1 2 17.098 6.853 14.502 0.91651470 0.023681576 0.91651470 0.059803724
44 44 1 2 18.608 14.268 18.301 0.68982183 0.146602447 0.68982183 0.163575721
45 45 1 1 11.059 10.812 20.121 0.46914173 0.469141731 0.51236934 0.018488932
46 46 1 2 15.641 10.754 24.669 0.84615894 0.147939578 0.84615894 0.005901485
47 47 1 1 7.822 18.949 16.904 0.94518707 0.945187070 0.01782790 0.036985032
48 48 1 2 12.824 5.697 19.183 0.92018353 0.072337036 0.92018353 0.007479430
49 49 1 2 11.852 12.147 15.672 0.41749018 0.463835709 0.41749018 0.118674106
50 50 1 2 15.557 8.307 22.286 0.92416940 0.069530571 0.92416940 0.0063000286.12.5 Multivariate Probit
Model 6. The constraints zero the terms that are not wanted.
Box 125Multivariate probit (qr6.cf)
mvp <- tda_qreg(Y ~ X, data = qr3, model = 6, nq = 5, weights = "Weight",
constraints = c("b9 = 0", "b10 = 0", "b11 = 0", "b12 = 0", "b13 = 0",
"b14 = 0", "b15 = 0", "b16 = 0", "b17 = 0",
"b18 = 0"))
mvpCall: tda_qreg(formula = Y ~ X, data = qr3, model = 6, nq = 5, constraints = c("b9 = 0",
"b10 = 0", "b11 = 0", "b12 = 0", "b13 = 0", "b14 = 0", "b15 = 0",
"b16 = 0", "b17 = 0", "b18 = 0"), weights = "Weight")
Cases: 45
Model: multivariate_probit (TDA 6)
logLik (starting values): -935.1167
logLik: -828.232
Converged in 5 iterations
Idx Cat Term Variable Coeff Error C/Error Signif
1 2.0000 I Intercept 2.8003 0.5528 5.0655 1.0000
2 2.0000 X X -0.1915 0.0431 -4.4477 1.0000
3 3.0000 I Intercept 3.3703 0.5454 6.1790 1.0000
4 3.0000 X X -0.2320 0.0425 -5.4523 1.0000
5 4.0000 I Intercept 4.8489 0.5609 8.6451 1.0000
6 4.0000 X X -0.3740 0.0445 -8.3994 1.0000
7 5.0000 I Intercept 4.5686 0.6054 7.5467 1.0000
8 5.0000 X X -0.3867 0.0493 -7.8424 1.0000
9 NA S Sigma 2, 1 0.0000 0.0000 NA NA
10 NA S Sigma 3, 1 0.0000 0.0000 NA NA
11 NA S Sigma 3, 2 0.0000 0.0000 NA NA
12 NA S Sigma 4, 1 0.0000 0.0000 NA NA
13 NA S Sigma 4, 2 0.0000 0.0000 NA NA
14 NA S Sigma 4, 3 0.0000 0.0000 NA NA
15 NA S Sigma 5, 1 0.0000 0.0000 NA NA
16 NA S Sigma 5, 2 0.0000 0.0000 NA NA
17 NA S Sigma 5, 3 0.0000 0.0000 NA NA
18 NA S Sigma 5, 4 0.0000 0.0000 NA NAunlist(mvp$stats)ncase
45 Box 126Multivariate probit over lvl() columns (qr7.cf)
mv <- tda_qreg(Y ~ 0 + lvl(Z1, Z2, Z3), data = qr4, model = 6, nq = 3,
start = c(-0.1716, 0, 0, 0), constraints = c("b3 = 0", "b4 = 0"))
mvCall: tda_qreg(formula = Y ~ 0 + lvl(Z1, Z2, Z3), data = qr4, model = 6,
nq = 3, constraints = c("b3 = 0", "b4 = 0"), start = c(-0.1716,
0, 0, 0))
Cases: 50
Model: multivariate_probit (TDA 6)
logLik (starting values): -34.5825
logLik: -34.41663
Converged in 4 iterations
Idx Cat Term Variable Coeff Error C/Error Signif
1 NA Z1 Lvl1_1 -0.1787 0.0349 -5.1178 1.0000
2 NA S Sigma 2, 1 0.4134 0.6320 0.6542 0.4870
3 NA S Sigma 3, 1 0.0000 0.0000 NA NA
4 NA S Sigma 3, 2 0.0000 0.0000 NA NA# the correlation among the latent equations, which TDA writes into its
# ppar= file with the rows commented
mv$correlation [,1] [,2] [,3]
[1,] 1.000000 0.413439 0
[2,] 0.413439 1.000000 0
[3,] 0.000000 0.000000 16.14.1 Poisson Regression
poisson(mu, y) is TDA’s log Poisson density, so a count
model is a likelihood like any other. LOGS becomes an
offset once its coefficient is constrained to one.
Box 127Poisson regression, free and with an offset (cd1.cf)
cd1 <- read.table(ex("cd1.dat"),
col.names = c("NDI", "Service", "B", "C", "D", "E",
"C60", "C65", "C70", "P75"))
cd1$LOGS <- log(cd1$Service)
cd1 <- cd1[cd1$NDI >= 0, ]
pois <- function(...) tda_fml({
xb = b0 + B * bb + C * bc + D * bd + E * be +
C60 * bc60 + C65 * bc65 + C70 * bc70 + P75 * bp75 + LOGS * blogs
exb = exp(xb)
fn = poisson(exb, NDI)
}, data = cd1, ...)
free <- pois()
freeCall: tda_fml(definitions = {
xb = b0 + B * bb + C * bc + D * bd + E * be + C60 * bc60 +
C65 * bc65 + C70 * bc70 + P75 * bp75 + LOGS * blogs
exb = exp(xb)
fn = poisson(exb, NDI)
}, data = cd1)
Cases: 34
logLik (starting values): -870.2939
logLik: -67.83538
Converged in 8 iterations
Idx Parameter Coeff Error C/Error Signif
1 b0 -5.5940 0.8724 -6.4121 1.0000
2 bb -0.3499 0.2702 -1.2948 0.8046
3 bc -0.7631 0.3382 -2.2565 0.9760
4 bc60 0.6625 0.1536 4.3124 1.0000
5 bc65 0.7597 0.1777 4.2763 1.0000
6 bc70 0.3697 0.2458 1.5040 0.8674
7 bd -0.1355 0.2971 -0.4560 0.3516
8 be 0.2739 0.2418 1.1331 0.7428
9 blogs 0.9027 0.1018 8.8672 1.0000
10 bp75 0.3703 0.1181 3.1339 0.9983# what the manual's box reports around the estimates
unlist(free$convergence) converged iterations problem final_change
"TRUE" "8" NA NA
gradient change.f change.par evaluations
"3.04826456175064e-11" "5.57244259068293e-14" "2.0557657693741e-06" "12" pois(constraints = "b9 = 1")Call: tda_fml(definitions = {
xb = b0 + B * bb + C * bc + D * bd + E * be + C60 * bc60 +
C65 * bc65 + C70 * bc70 + P75 * bp75 + LOGS * blogs
exb = exp(xb)
fn = poisson(exb, NDI)
}, data = cd1, constraints = "b9 = 1")
Cases: 34
logLik (starting values): -161157.8
logLik: -68.28077
Converged in 12 iterations
Idx Parameter Coeff Error C/Error Signif
1 b0 -6.4059 0.2174 -29.4600 1.0000
2 bb -0.5433 0.1776 -3.0595 0.9978
3 bc -0.6874 0.3290 -2.0891 0.9633
4 bc60 0.6971 0.1496 4.6587 1.0000
5 bc65 0.8184 0.1698 4.8207 1.0000
6 bc70 0.4534 0.2332 1.9446 0.9482
7 bd -0.0760 0.2906 -0.2614 0.2062
8 be 0.3256 0.2359 1.3803 0.8325
9 blogs 1.0000 NA NA NA
10 bp75 0.3845 0.1183 3.2507 0.99886.14.2 Compound Poisson Models
cd2.cf is the same count model with a gamma-distributed
rate – the negative binomial, negbin(alpha, gamma, y) in
TDA’s expression language – and mina = 8 for a more robust
algorithm. The overdispersion parameter sig is not
identified on these data: its error is two orders of magnitude larger
than its value, and its last digits depend on where the algorithm stops
(the manual’s box reads -14.77, this run -15.06); the regression
coefficients are the manual’s.
Box 128cd2.cf: a negative binomial count model
tda_fml({
xb = b0 + B * bb + C * bc + D * bd + E * be +
C60 * bc60 + C65 * bc65 + C70 * bc70 + P75 * bp75 + LOGS * blogs
gamma = exp(xb)
sigma = exp(sig)
alpha = 1 / sigma
fn = negbin(alpha, gamma, NDI)
}, data = cd1, options = list(mina = 8))Call: tda_fml(definitions = {
xb = b0 + B * bb + C * bc + D * bd + E * be + C60 * bc60 +
C65 * bc65 + C70 * bc70 + P75 * bp75 + LOGS * blogs
gamma = exp(xb)
sigma = exp(sig)
alpha = 1/sigma
fn = negbin(alpha, gamma, NDI)
}, data = cd1, options = list(mina = 8))
Cases: 34
logLik (starting values): -270.3274
logLik: -67.8354
Converged in 17 iterations
Idx Parameter Coeff Error C/Error Signif
1 b0 -5.5940 0.8724 -6.4121 1.0000
2 bb -0.3499 0.2702 -1.2948 0.8046
3 bc -0.7631 0.3382 -2.2565 0.9760
4 bc60 0.6625 0.1536 4.3124 1.0000
5 bc65 0.7597 0.1777 4.2763 1.0000
6 bc70 0.3697 0.2458 1.5040 0.8674
7 bd -0.1355 0.2971 -0.4560 0.3516
8 be 0.2739 0.2418 1.1331 0.7428
9 blogs 0.9027 0.1018 8.8672 1.0000
10 bp75 0.3703 0.1181 3.1339 0.9983
11 sig -15.0638 212.9850 -0.0707 0.05646.15.2.1 Generalized Linear Models
tda_glm() is TDA’s glm. Without a family it
is the Gaussian model with the identity link, which is least squares;
link = 2 is the log link, which needs starting values
here.
Box 129glm, Gaussian, identity and log links
plain <- tda_glm(Weight ~ Height, data = lsreg1)
plainCall: tda_glm(formula = Weight ~ Height, data = lsreg1)
Cases: 19
Family: gaussian
Link: identity
Converged in 2 iterations
Idx Wave Variable Coeff Error C/Error Signif
1 NA Intercept -143.0269 32.2746 -4.4316 0.9996
2 1.0000 Height 3.8990 0.5161 7.5549 1.0000# the manual prints the deviance as 2142.4880. Reading its data file TDA
# stores each variable as a 4-byte float; the wrapper hands R's doubles
# over directly, and stats::glm on the same data agrees with it exactly.
unlist(plain$stats) df deviance pearson scale.ml scale.deviance rank
17.0000 2142.4877 2142.4877 112.7625 126.0287 2.0000 gen <- tda_rng()
n <- 100
X <- Y <- numeric(n)
for (i in seq_len(n)) {
X[i] <- gen$rd(a = -3, b = 3)
Y[i] <- exp(3 + X[i]) + gen$rd()
}
bad <- tda_glm(Y ~ X, data = data.frame(X, Y), start = c(5, 5), link = 2)
badCall: tda_glm(formula = Y ~ X, data = data.frame(X, Y), link = 2, start = c(5,
5))
Cases: 100
Family: gaussian
Link: log
Converged in 19 iterations
Idx Wave Variable Coeff Error C/Error Signif
1 NA Intercept 3.0128 0.0020 1524.0437 1.0000
2 1.0000 X 0.9958 0.0008 1299.1796 1.0000# what the manual's Box 4 reports around the estimates: the degrees of
# freedom, the deviance and Pearson statistics, the scale estimates and
# the rank, and how far the last step moved
unlist(bad$stats) df deviance pearson scale.ml scale.deviance rank
98.0000000 19.9937532 19.9937532 0.1999375 0.2040179 2.0000000 unlist(bad$convergence)[c("converged", "iterations", "final_change")] converged iterations final_change
"TRUE" "19" "4.13312e-10" 6.15.2.2 Binomial Models
The number of trials is given separately, and a link can be written
out rather than selected: custom_link= takes a TDA
expression in mue.
Box 130Binomial models, selected and hand-written links
glmbin1 <- tda_glm(Response ~ Log10Dose, data = qr1, weights = "Weight", family = 2)
glmbin1Call: tda_glm(formula = Response ~ Log10Dose, data = qr1, family = 2,
weights = "Weight")
Cases: 13
Family: binomial
Link: logit
Converged in 6 iterations
Idx Wave Variable Coeff Error C/Error Signif
1 NA Intercept -3.2246 0.8861 -3.6393 0.9961
2 1.0000 Log10Dose 5.9702 1.4492 4.1197 0.9983unlist(glmbin1$stats) df deviance rank
11.00000 74.22131 2.00000 glm3 <- read.table(ex("glm3.dat"), col.names = c("D", "N", "Y"))
glm3$D <- log(glm3$D)
glmbin2 <- tda_glm(Y ~ D, data = glm3, family = 2, link = 3, trials = glm3$N)
glmbin2Call: tda_glm(formula = Y ~ D, data = glm3, family = 2, link = 3, trials = glm3$N)
Cases: 6
Family: binomial
Link: logit
Converged in 8 iterations
Idx Wave Variable Coeff Error C/Error Signif
1 NA Intercept -4.8869 0.6429 -7.6010 0.9984
2 1.0000 D 3.1035 0.3877 8.0047 0.9987unlist(glmbin2$stats) df deviance rank
4.000000 1.424057 2.000000 glmbin1 <- tda_glm(Y ~ D, data = glm3, family = 2, trials = glm3$N,
custom_link = "log(mue / (1 - mue))")
glmbin1Call: tda_glm(formula = Y ~ D, data = glm3, family = 2, custom_link = "log(mue / (1 - mue))",
trials = glm3$N)
Cases: 6
Family: binomial
Converged in 8 iterations
Idx Wave Variable Coeff Error C/Error Signif
1 NA Intercept -4.8869 0.6429 -7.6010 0.9984
2 1.0000 D 3.1035 0.3877 8.0047 0.9987unlist(glmbin1$stats) df deviance rank
4.000000 1.424057 2.000000 6.15.2.3 Poisson Models
The offset is the same constrained coefficient as in 6.14.1, given here as an equality constraint on the GLM.
Box 131The Poisson family
glm5 <- read.table(ex("glm5.dat"), col.names = c("X", "Y"))
glmpois1 <- tda_glm(Y ~ X, data = glm5, family = "poisson")
glmpois1Call: tda_glm(formula = Y ~ X, data = glm5, family = "poisson")
Cases: 10
Family: poisson
Link: log
Converged in 9 iterations
Idx Wave Variable Coeff Error C/Error Signif
1 NA Intercept 0.8673 0.2384 3.6382 0.9934
2 1.0000 X 1.1754 0.3565 3.2974 0.9891unlist(glmpois1$stats) df deviance rank
8.000000 6.190151 2.000000 glmpois1 <- tda_glm(NDI ~ B + C + D + E + C60 + C65 + C70 + P75 + LOGS, data = cd1,
family = "poisson", equality = c("b9 = 1"),
control = tda_control(maxit = 100))
glmpois1Call: tda_glm(formula = NDI ~ B + C + D + E + C60 + C65 + C70 + P75 +
LOGS, data = cd1, family = "poisson", equality = c("b9 = 1"),
control = tda_control(maxit = 100))
Cases: 34
Family: poisson
Link: log
Converged in 41 iterations
Idx Wave Variable Coeff Error C/Error Signif
1 NA Intercept -6.4059 0.2174 -29.4600 1.0000
2 1.0000 B -0.5433 0.1776 -3.0595 0.9948
3 1.0000 C -0.6874 0.3290 -2.0891 0.9530
4 1.0000 D -0.0760 0.2906 -0.2614 0.2041
5 1.0000 E 0.3256 0.2359 1.3803 0.8203
6 1.0000 C60 0.6971 0.1496 4.6587 0.9999
7 1.0000 C65 0.8184 0.1698 4.8207 0.9999
8 1.0000 C70 0.4534 0.2332 1.9446 0.9368
9 1.0000 P75 0.3845 0.1183 3.2507 0.9967
10 1.0000 LOGS 1.0000 NA NA NAunlist(glmpois1$stats) df deviance rank
25.00000 38.69505 9.00000 6.15.2.4 Gamma Models
Box 132The Gamma family, log and inverse links
gen <- tda_rng()
n <- 1000
X <- Y <- numeric(n)
for (i in seq_len(n)) {
X[i] <- gen$rd()
Y[i] <- -exp(1 + X[i]) * (log(gen$rd()) + log(gen$rd())) / 2
}
glmgamma1 <- tda_glm(Y ~ X, data = data.frame(X, Y), family = "Gamma", link = 2,
start = c(1, 1), control = tda_control(maxit = 50))
glmgamma1Call: tda_glm(formula = Y ~ X, data = data.frame(X, Y), family = "Gamma",
link = 2, start = c(1, 1), control = tda_control(maxit = 50))
Cases: 1000
Family: Gamma
Link: log
Converged in 4 iterations
Idx Wave Variable Coeff Error C/Error Signif
1 NA Intercept 0.9620 0.0624 15.4239 1.0000
2 1.0000 X 1.0947 0.1070 10.2332 1.0000unlist(glmgamma1$stats) df deviance rank
998.0000 519.1471 2.0000 # glm7.cf's second model: the same generator continued, with expectation
# 1/(1+X), and the gamma family's canonical (reciprocal) link
X2 <- Y2 <- numeric(n)
for (i in seq_len(n)) {
X2[i] <- gen$rd()
Y2[i] <- -(log(gen$rd()) + log(gen$rd())) / ((1 + X2[i]) * 2)
}
glmgamma2 <- tda_glm(Y2 ~ X2, data = data.frame(X2, Y2), family = "Gamma",
start = c(1, 1), control = tda_control(maxit = 50))
glmgamma2Call: tda_glm(formula = Y2 ~ X2, data = data.frame(X2, Y2), family = "Gamma",
start = c(1, 1), control = tda_control(maxit = 50))
Cases: 1000
Family: Gamma
Link: inverse
Converged in 3 iterations
Idx Wave Variable Coeff Error C/Error Signif
1 NA Intercept 1.0082 0.0748 13.4819 1.0000
2 1.0000 X2 1.0223 0.1604 6.3736 1.0000unlist(glmgamma2$stats) df deviance rank
998.0000 550.6416 2.0000 glm8a <- read.table(ex("glm8a.dat"), col.names = c("U", "Y", "L"))
glm8a$X <- log(glm8a$U)
glmgamma3 <- tda_glm(Y ~ X, data = glm8a[glm8a$L == 0, ], family = "Gamma",
start = c(-0.02, 0.02))
glmgamma3Call: tda_glm(formula = Y ~ X, data = glm8a[glm8a$L == 0, ], family = "Gamma",
start = c(-0.02, 0.02))
Cases: 9
Family: Gamma
Link: inverse
Converged in 5 iterations
Idx Wave Variable Coeff Error C/Error Signif
1 NA Intercept -0.0166 0.0188 -0.8827 0.5933
2 1.0000 X 0.0153 0.0084 1.8287 0.8898unlist(glmgamma3$stats) df deviance rank
7.00000000 0.01672972 2.00000000 glmgamma4 <- tda_glm(Y ~ X, data = glm8a[glm8a$L == 1, ], family = "Gamma",
start = c(-0.02, 0.02))
glmgamma4Call: tda_glm(formula = Y ~ X, data = glm8a[glm8a$L == 1, ], family = "Gamma",
start = c(-0.02, 0.02))
Cases: 9
Family: Gamma
Link: inverse
Converged in 4 iterations
Idx Wave Variable Coeff Error C/Error Signif
1 NA Intercept -0.0239 0.0311 -0.7675 0.5321
2 1.0000 X 0.0236 0.0135 1.7423 0.8750unlist(glmgamma4$stats) df deviance rank
7.00000000 0.01267176 2.00000000 6.15.2.5 Inverse Gaussian Models
glm9.cf draws from an inverse Gaussian with mean 1 + X
through TDA’s own generator, and fits it with the identity link from the
starting values 0.5, 0.5.
Box 133The inverse Gaussian family (glm9.cf)
gen <- tda_rng()
n <- 1000
X <- Mu <- Z <- X1 <- Y <- numeric(n)
for (i in seq_len(n)) {
X[i] <- gen$rd()
Mu[i] <- 1 + X[i]
Z[i] <- gen$rdn()^2
X1[i] <- Mu[i] + Mu[i]^2 * Z[i] -
Mu[i] * sqrt(2 * Mu[i] * Z[i] + Mu[i]^2 * Z[i]^2)
Y[i] <- ifelse(gen$rd() <= Mu[i] / (Mu[i] + X1[i]), X1[i], Mu[i]^2 / X1[i])
}
ig <- tda_glm(Y ~ X, data = data.frame(X, Y), family = "inverse.gaussian",
link = 1, start = c(0.5, 0.5))
igCall: tda_glm(formula = Y ~ X, data = data.frame(X, Y), family = "inverse.gaussian",
link = 1, start = c(0.5, 0.5))
Cases: 1000
Family: inverse.gaussian
Link: identity
Converged in 6 iterations
Idx Wave Variable Coeff Error C/Error Signif
1 NA Intercept 1.0702 0.0867 12.3497 1.0000
2 1.0000 X 0.8148 0.1909 4.2690 1.0000unlist(ig$stats) df deviance rank
998.000 1933.998 2.000 unlist(ig$convergence)[c("converged", "iterations", "final_change")] converged iterations final_change
"TRUE" "6" "7.00739e-09" 6.16.1 The freg Command
tda_freg() is nonlinear least squares: the function
block gives the model and fn the squared residual, summed
over the data. The manual’s example is Gallant’s (1987, p. 4) data, 30
observations, which the manual’s freg1.dat holds; they are
typed in here.
Box 134freg: Gallant’s example (freg1.cf)
freg1dat <- read.csv(ex("gallant.dat"))
tda_freg({
ax = a1 * X1 + a2 * X2 + a4 * exp(a3 * X3)
fn = (Y - ax)^2
}, data = freg1dat, start = c(-0.1, 1, -1, -1),
control = tda_control(algorithm = 5))Call: tda_freg(definitions = {
ax = a1 * X1 + a2 * X2 + a4 * exp(a3 * X3)
fn = (Y - ax)^2
}, data = freg1dat, start = c(-0.1, 1, -1, -1), control = tda_control(algorithm = 5))
Cases: 30
Converged in 9 iterations
Idx Parameter Coeff Error C/Error Signif
1 a1 -0.0259 0.0126 -2.0511 0.9597
2 a2 1.0157 0.0099 102.4480 1.0000
3 a3 -1.1157 0.1607 -6.9423 1.0000
4 a4 -0.5049 0.0255 -19.7862 1.00006.17.2.1 The Basic Exponential Model
The simplest transition rate model lets the rate vary with the
covariates but not with the duration of the episode. The duration is
exponentially distributed, with density f(t) = a exp(-a t),
survivor function G(t) = exp(-a t) and rate
r(t) = a, and the covariates enter through an exponential
link,
r(jk) = exp(A(jk) alpha(jk)),
where A holds a constant one in its first component for
the intercept. This is TDA’s model number 2. The
xa(0,1) = ... parameter of the rate command,
which links covariates to the transition from 0 to 1, is the right hand
side of the formula.
Example 1 The manual’s rt1.cf links
three covariates to the only transition in the data.
Box 135The command file rt1.cf, in R
fit <- tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 2,
residuals = TRUE,
# rt1.cf's constellation: COHO3 = 1 and W = 1
prate = list(tp = seq(0, 100, 5), COHO3 = 1, W = 1))
fitCall: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 2, prate = list(tp = seq(0, 100, 5), COHO3 = 1, W = 1),
residuals = TRUE)
Episodes: 600
Model: 2 (TDA 2)
logLik (starting values): -2514.02
logLik: -2475.438
Converged in 5 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Constant -5.0114 0.0843 -59.4446 1.0000
2 1 0 1 A COHO2 0.5341 0.1120 4.7686 1.0000
3 1 0 1 A COHO3 0.6738 0.1152 5.8472 1.0000
4 1 0 1 A W 0.5065 0.0942 5.3746 1.0000rt1.cf asks for two things besides the estimates: the
generalized residuals for every episode, and the fitted rate over a time
axis at a chosen covariate constellation.
Box 136rt1.cf: generalized residuals and the rate table
# the manual prints part of res.d: its first rows and its last
head(fit$residuals, 6) Case Org Des TS TF Rate Function Residual Weight
1 1 0 0 0 428 0.006661875 0.05777019 2.8512823 1
2 2 0 1 0 46 0.011055526 0.60136441 0.5085542 1
3 3 0 1 0 34 0.011055526 0.68667932 0.3758879 1
4 4 0 1 0 220 0.011055526 0.08784199 2.4322157 1
5 5 0 1 0 12 0.018859457 0.79746807 0.2263135 1
6 6 0 1 0 30 0.018859457 0.56791490 0.5657837 1tail(fit$residuals, 6) Case Org Des TS TF Rate Function Residual Weight
595 595 0 1 0 119 0.006661875 0.4525925 0.7927631 1
596 596 0 1 0 66 0.006661875 0.6442401 0.4396837 1
597 597 0 0 0 112 0.006661875 0.4741982 0.7461300 1
598 598 0 0 0 103 0.013068795 0.2602569 1.3460859 1
599 599 0 1 0 22 0.018859457 0.6604010 0.4149080 1
600 600 0 1 0 13 0.018859457 0.7825692 0.2451729 1# the rate table, and the constellation it was computed at -- which
# covariate values, and the coefficients they multiply
attr(fit$rates, "constellation") idx sn org des mt variable coeff covariate
1 1 1 0 1 A Constant -5.0113544 1
2 2 1 0 1 A COHO2 0.5340841 0
3 3 1 0 1 A COHO3 0.6738264 1
4 4 1 0 1 A W 0.5065294 1fit$rates ID Time Surv.F Density Rate
1 0 0 1.0000000 0.021687950 0.02168795
2 0 5 0.8972330 0.019459143 0.02168795
3 0 10 0.8050270 0.017459385 0.02168795
4 0 15 0.7222967 0.015665135 0.02168795
5 0 20 0.6480684 0.014055276 0.02168795
6 0 25 0.5814683 0.012610856 0.02168795
7 0 30 0.5217126 0.011314876 0.02168795
8 0 35 0.4680977 0.010152079 0.02168795
9 0 40 0.4199927 0.009108780 0.02168795
10 0 45 0.3768313 0.008172698 0.02168795
11 0 50 0.3381054 0.007332814 0.02168795
12 0 55 0.3033593 0.006579242 0.02168795
13 0 60 0.2721840 0.005903113 0.02168795
14 0 65 0.2442124 0.005296467 0.02168795
15 0 70 0.2191155 0.004752165 0.02168795
16 0 75 0.1965976 0.004263799 0.02168795
17 0 80 0.1763939 0.003825621 0.02168795
18 0 85 0.1582664 0.003432473 0.02168795
19 0 90 0.1420018 0.003079728 0.02168795
20 0 95 0.1274087 0.002763234 0.02168795
21 0 100 0.1143153 0.002479264 0.02168795Convergence is reached in 5 iterations with TDA’s default algorithm,
and the coefficients are the ones printed in the manual. They are not
read back from that printout: coef() holds the doubles TDA
computed.
Box 137The estimates, at full precision
coef(fit) Constant COHO2 COHO3 W
-5.0113544 0.5340841 0.6738264 0.5065294 logLik(fit)'log Lik.' -2475.438 (df=4)fit$logLik_null # the exponential null model TDA starts from[1] -2514.02Example 2 rt1m.cf estimates the same
three covariates over the three destination states. Its DES
differs from ed2.cf’s in one comparison –
ge(PRESN/PRES - 1, 0.2) against gt – and one
episode lies on 0.2 to within TDA’s comparison tolerance, so the
manual’s episode table here reads 85/218 where 3.3.3’s read 84/219;
tda_rrdat(upward = "ge") is that coding.
relative_risk= adds the manual’s table of rate ratios.
Box 138rt1m.cf: three transitions, and the relative risks (frml1m.cf)
d4m <- tda_rrdat(states = 4, upward = "ge")
d4m$W <- as.integer(d4m$SEX == 2)
fitm <- tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d4m, model = 2,
relative_risk = TRUE)
fitm$episodes sn org des episodes weighted mean.duration ts.min tf.max excluded
1 1 0 0 142 142 128.17606 0 428 FALSE
2 1 0 1 85 85 44.81176 0 326 FALSE
3 1 0 2 218 218 46.00917 0 350 FALSE
4 1 0 3 155 155 56.40000 0 332 FALSEfitm$estimates Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 1 0 1 A Constant -6.34115531 0.1736214 -36.5228896 1.0000000
2 2 1 0 1 A COHO2 0.30116538 0.2617898 1.1504092 0.7500246
3 3 1 0 1 A COHO3 0.49987105 0.2650832 1.8857137 0.9406665
4 4 1 0 1 A W -0.09566548 0.2342343 -0.4084179 0.3170331
5 5 1 0 2 A Constant -5.75478094 0.1236419 -46.5439221 1.0000000
6 6 1 0 2 A COHO2 0.69241033 0.1629726 4.2486308 0.9999785
7 7 1 0 2 A COHO3 0.80879929 0.1683635 4.8038877 0.9999984
8 8 1 0 2 A W 0.29771709 0.1381110 2.1556362 0.9688879
9 9 1 0 3 A Constant -6.36856915 0.1577556 -40.3698331 1.0000000
10 10 1 0 3 A COHO2 0.44239916 0.1923777 2.2996383 0.9785313
11 11 1 0 3 A COHO3 0.58747938 0.1979632 2.9676193 0.9969988
12 12 1 0 3 A W 1.11561053 0.1658799 6.7254092 1.0000000fitmCall: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d4m,
model = 2, relative_risk = TRUE)
Episodes: 600
Model: 2 (TDA 2)
logLik (starting values): -2986.95
logLik: -2935.583
Converged in 5 iterations
Transitions: 0->1, 0->2, 0->3
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Constant -6.3412 0.1736 -36.5229 1.0000
2 1 0 1 A COHO2 0.3012 0.2618 1.1504 0.7500
3 1 0 1 A COHO3 0.4999 0.2651 1.8857 0.9407
4 1 0 1 A W -0.0957 0.2342 -0.4084 0.3170
5 1 0 2 A Constant -5.7548 0.1236 -46.5439 1.0000
6 1 0 2 A COHO2 0.6924 0.1630 4.2486 1.0000
7 1 0 2 A COHO3 0.8088 0.1684 4.8039 1.0000
8 1 0 2 A W 0.2977 0.1381 2.1556 0.9689
9 1 0 3 A Constant -6.3686 0.1578 -40.3698 1.0000
10 1 0 3 A COHO2 0.4424 0.1924 2.2996 0.9785
11 1 0 3 A COHO3 0.5875 0.1980 2.9676 0.9970
12 1 0 3 A W 1.1156 0.1659 6.7254 1.0000fitm$relative_risk Idx SN Org Des MT Variable R.Risk
1 1 1 0 1 A Constant 0.001762265
2 2 1 0 1 A COHO2 1.351432826
3 3 1 0 1 A COHO3 1.648508683
4 4 1 0 1 A W 0.908767967
5 5 1 0 2 A Constant 0.003167600
6 6 1 0 2 A COHO2 1.998526843
7 7 1 0 2 A COHO3 2.245210517
8 8 1 0 2 A W 1.346780721
9 9 1 0 3 A Constant 0.001714611
10 10 1 0 3 A COHO2 1.556436885
11 11 1 0 3 A COHO3 1.799446975
12 12 1 0 3 A W 3.0514305956.17.2.2 Models with Time Periods
Model 3 is the piecewise constant exponential: a separate rate in
each period given by tp=, the covariate effects held
common.
Box 139A piecewise constant exponential (rt2.cf)
rt2 <- tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 3,
tp = seq(0, 96, 12))
rt2$periods start end starting ending events
1 0 12 600 76 63
2 12 24 0 104 96
3 24 36 0 108 95
4 36 48 0 55 44
5 48 60 0 44 38
6 60 72 0 38 29
7 72 84 0 22 17
8 84 96 0 9 6
9 96 Inf 0 144 70rt2Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 3, tp = seq(0, 96, 12))
Episodes: 600
Model: 3 (TDA 3)
logLik (starting values): -2514.02
logLik: -2433.376
Converged in 5 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Period-1 -5.2070 0.1520 -34.2464 1.0000
2 1 0 1 A Period-2 -4.5473 0.1312 -34.6574 1.0000
3 1 0 1 A Period-3 -4.2675 0.1296 -32.9389 1.0000
4 1 0 1 A Period-4 -4.7784 0.1690 -28.2758 1.0000
5 1 0 1 A Period-5 -4.7300 0.1779 -26.5924 1.0000
6 1 0 1 A Period-6 -4.7862 0.1987 -24.0857 1.0000
7 1 0 1 A Period-7 -5.1366 0.2523 -20.3568 1.0000
8 1 0 1 A Period-8 -6.0783 0.4137 -14.6918 1.0000
9 1 0 1 A Period-9 -5.4198 0.1281 -42.3123 1.0000
10 1 0 1 A COHO2 0.4367 0.1133 3.8534 0.9999
11 1 0 1 A COHO3 0.5162 0.1182 4.3688 1.0000
12 1 0 1 A W 0.4336 0.0949 4.5688 1.0000rt2c.cf constrains all nine period coefficients to be
equal, which gives the exponential model of rt1.cf back –
the manual’s way of showing what a constraint does.
Box 140rt2c.cf: all periods constrained equal
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 3,
tp = seq(0, 96, 12),
constraints = sprintf("b%d - b%d = 0", 1:8, 2:9))Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 3, constraints = sprintf("b%d - b%d = 0", 1:8, 2:9),
tp = seq(0, 96, 12))
Episodes: 600
Model: 3 (TDA 3)
logLik (starting values): -2514.02
logLik: -2475.438
Converged in 5 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Period-1 -5.0114 0.0843 -59.4446 1.0000
2 1 0 1 A Period-2 -5.0114 0.0843 -59.4446 1.0000
3 1 0 1 A Period-3 -5.0114 0.0843 -59.4446 1.0000
4 1 0 1 A Period-4 -5.0114 0.0843 -59.4446 1.0000
5 1 0 1 A Period-5 -5.0114 0.0843 -59.4446 1.0000
6 1 0 1 A Period-6 -5.0114 0.0843 -59.4446 1.0000
7 1 0 1 A Period-7 -5.0114 0.0843 -59.4446 1.0000
8 1 0 1 A Period-8 -5.0114 0.0843 -59.4446 1.0000
9 1 0 1 A Period-9 -5.0114 0.0843 -59.4446 1.0000
10 1 0 1 A COHO2 0.5341 0.1120 4.7686 1.0000
11 1 0 1 A COHO3 0.6738 0.1152 5.8472 1.0000
12 1 0 1 A W 0.5065 0.0942 5.3746 1.00006.17.2.3 Period-specific Effects
Model 16 gives each period its covariate effects as well as its own baseline.
Box 141Period-specific effects (rt3.cf)
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 16,
tp = seq(0, 96, 12))Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 16, tp = seq(0, 96, 12))
Episodes: 600
Model: 16 (TDA 16)
logLik (starting values): -2514.02
logLik: -2411.991
Converged in 10 iterations
Idx SN Org Des MT P Variable Coeff Error C/Error Signif
1 1 0 1 A 1.0000 Period-1 -5.0470 0.2696 -18.7220 1.0000
2 1 0 1 A 2.0000 Period-2 -4.5147 0.2121 -21.2865 1.0000
3 1 0 1 A 3.0000 Period-3 -4.1413 0.1949 -21.2431 1.0000
4 1 0 1 A 4.0000 Period-4 -5.1107 0.3160 -16.1711 1.0000
5 1 0 1 A 5.0000 Period-5 -4.9954 0.3226 -15.4869 1.0000
6 1 0 1 A 6.0000 Period-6 -4.5366 0.2934 -15.4603 1.0000
7 1 0 1 A 7.0000 Period-7 -5.5069 0.5127 -10.7403 1.0000
8 1 0 1 A 8.0000 Period-8 -6.0808 0.7231 -8.4090 1.0000
9 1 0 1 A 9.0000 Period-9 -5.5356 0.1732 -31.9616 1.0000
10 1 0 1 A 1.0000 COHO2 0.4474 0.3362 1.3308 0.8168
11 1 0 1 A 2.0000 COHO2 0.5783 0.2509 2.3050 0.9788
12 1 0 1 A 3.0000 COHO2 0.4240 0.2411 1.7581 0.9213
13 1 0 1 A 4.0000 COHO2 0.2850 0.3616 0.7881 0.5694
14 1 0 1 A 5.0000 COHO2 0.6357 0.4089 1.5546 0.8800
15 1 0 1 A 6.0000 COHO2 -0.0546 0.4292 -0.1273 0.1013
16 1 0 1 A 7.0000 COHO2 1.5141 0.6137 2.4671 0.9864
17 1 0 1 A 8.0000 COHO2 -0.8695 1.0971 -0.7925 0.5719
18 1 0 1 A 9.0000 COHO2 0.2966 0.3020 0.9822 0.6740
19 1 0 1 A 1.0000 COHO3 0.7676 0.3161 2.4284 0.9848
20 1 0 1 A 2.0000 COHO3 0.5335 0.2609 2.0446 0.9591
21 1 0 1 A 3.0000 COHO3 0.2488 0.2590 0.9606 0.6633
22 1 0 1 A 4.0000 COHO3 0.3212 0.3684 0.8717 0.6166
23 1 0 1 A 5.0000 COHO3 0.8420 0.3946 2.1337 0.9671
24 1 0 1 A 6.0000 COHO3 -0.7163 0.5605 -1.2779 0.7987
25 1 0 1 A 7.0000 COHO3 1.1975 0.6762 1.7708 0.9234
26 1 0 1 A 8.0000 COHO3 -8.4412 42.1447 -0.2003 0.1587
27 1 0 1 A 9.0000 COHO3 1.2489 0.3227 3.8701 0.9999
28 1 0 1 A 1.0000 W -0.1421 0.2620 -0.5422 0.4123
29 1 0 1 A 2.0000 W 0.2518 0.2054 1.2259 0.7798
30 1 0 1 A 3.0000 W 0.3492 0.2060 1.6958 0.9101
31 1 0 1 A 4.0000 W 1.2122 0.3188 3.8026 0.9999
32 1 0 1 A 5.0000 W 0.6285 0.3255 1.9310 0.9465
33 1 0 1 A 6.0000 W 0.7757 0.3749 2.0691 0.9615
34 1 0 1 A 7.0000 W -0.2436 0.5366 -0.4539 0.3501
35 1 0 1 A 8.0000 W 1.7909 0.8673 2.0650 0.9611
36 1 0 1 A 9.0000 W 0.5607 0.2580 2.1729 0.97026.17.3 Parametric Duration Dependence
The parametric family is a matter of the model number, and
TDA_MODELS names them, so model = "weibull"
and model = 7 are the same request. Each family has its
section in the manual, with the same three covariates as
rt1.cf; the boxes below are those sections’ models, one
each.
Box 142The model numbers
TDA_MODELS cox exponential exponential_periods polynomial
1 2 3 4
polynomial2 gompertz_makeham weibull sickle
5 6 7 8
loglogistic loglogistic2 loglogistic2a lognormal
9 10 11 12
generalized_gamma invgaussian exponential_periods2 discrete_logistic
13 14 16 20
discrete_cloglog
21 6.17.3.1 Polynomial Rates
Models 4 and 5, with degree=: the rate, or its
logarithm, a polynomial in time.
Box 143Polynomial rates
poly1 <- tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 4, degree = 2)
poly1Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 4, degree = 2)
Episodes: 600
Model: 4 (TDA 4)
logLik (starting values): -2514.02
logLik: -2454.039
Converged in 7 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Constant -4.4325 0.1031 -42.9986 1.0000
2 1 0 1 A COHO2 0.2452 0.0893 2.7468 0.9940
3 1 0 1 A COHO3 0.3411 0.0991 3.4429 0.9994
4 1 0 1 A W 0.3175 0.0758 4.1878 1.0000
5 1 0 1 B Beta-1 -0.0001 0.0000 -4.8750 1.0000
6 1 0 1 B Beta-2 0.0000 0.0000 3.2809 0.9990unlist(poly1$convergence) # what the manual prints above the estimates converged iterations problem final_change
"TRUE" "7" NA NA
gradient change.f change.par evaluations
"0.00705957041140394" "9.30147686918993e-14" "2.72933217258955e-06" "10" poly2 <- tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 5, degree = 2)
poly2Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 5, degree = 2)
Episodes: 600
Model: 5 (TDA 5)
logLik (starting values): -2514.02
logLik: -2454.54
Converged in 7 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Constant -4.5911 0.1166 -39.3890 1.0000
2 1 0 1 A COHO2 0.3831 0.1132 3.3834 0.9993
3 1 0 1 A COHO3 0.4583 0.1178 3.8908 0.9999
4 1 0 1 A W 0.4173 0.0948 4.3996 1.0000
5 1 0 1 B Beta-1 -0.0037 0.0022 -1.7010 0.9111
6 1 0 1 B Beta-2 -0.0000 0.0000 -0.7764 0.5625unlist(poly2$convergence) converged iterations problem final_change
"TRUE" "7" NA NA
gradient change.f change.par evaluations
"5.24057885883376e-07" "3.43758764652829e-15" "3.81351790694824e-06" "8" 6.17.3.2 Gompertz-Makeham Models
Model 6. on= says which terms of the model the
covariates are linked to; the manual’s example puts them on the manual’s
xb and xc.
Box 144Gompertz-Makeham (rt6.cf)
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 6,
on = c("xb", "xc"))Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 6, on = c("xb", "xc"))
Episodes: 600
Model: 6 (TDA 6)
logLik (starting values): -2557.365
logLik: -2453.383
Converged in 6 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 B Constant -4.5114 0.1198 -37.6503 1.0000
2 1 0 1 B COHO2 0.4595 0.1546 2.9716 0.9970
3 1 0 1 B COHO3 0.4147 0.1657 2.5025 0.9877
4 1 0 1 B W 0.2860 0.1280 2.2351 0.9746
5 1 0 1 C Constant -0.0060 0.0013 -4.6239 1.0000
6 1 0 1 C COHO2 -0.0016 0.0022 -0.7275 0.5331
7 1 0 1 C COHO3 0.0011 0.0028 0.3986 0.3098
8 1 0 1 C W 0.0026 0.0018 1.4299 0.8472Box 145plot-gm.cf
p <- tda_ps(xlim = c(0, 5), ylim = c(0, 3), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(1, 1))
p <- tda_pl_frame(p)
p <- tda_pl_function(p, "1+0*x1", range = c(0, 5), step = 1)
p <- tda_pl_function(p, "exp(0.2*x1)", range = c(0, 5), step = 0.1, lty = 5)
p <- tda_pl_function(p, "exp(-0.5*x1)", range = c(0, 5), step = 0.1, lty = 6)
p <- tda_pl_text(p, "c = 0.0", at = c(4.3, 1.1))
p <- tda_pl_text(p, "c = 0.2", at = c(4.3, 2.1))
p <- tda_pl_text(p, "c = -0.5", at = c(4.3, 0.2))
plot(p)
6.17.3.3 Weibull Models
Box 146Weibull (rt7.cf)
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 7)Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 7)
Episodes: 600
Model: 7 (TDA 7)
logLik (starting values): -2514.02
logLik: -2473.543
Converged in 6 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Constant -5.0055 0.0906 -55.2452 1.0000
2 1 0 1 A COHO2 0.5339 0.1201 4.4448 1.0000
3 1 0 1 A COHO3 0.6671 0.1235 5.4003 1.0000
4 1 0 1 A W 0.5191 0.1012 5.1274 1.0000
5 1 0 1 B Constant -0.0700 0.0366 -1.9119 0.9441Box 147plot-wei.cf
wei <- function(b) sprintf("%s*1^%s*x1^(%s-1)", b, b, b)
p <- tda_ps(xlim = c(0, 3), ylim = c(0, 3), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(1, 1))
p <- tda_pl_frame(p)
p <- tda_pl_function(p, "1+0*x1", range = c(0.01, 3), step = 1)
p <- tda_pl_function(p, wei(1.2), range = c(0.01, 3), step = 0.01, lty = 5)
p <- tda_pl_function(p, wei(1.5), range = c(0.01, 3), step = 0.01, lty = 6)
p <- tda_pl_function(p, wei(0.7), range = c(0.01, 3), step = 0.01, lty = 5)
p <- tda_pl_function(p, wei(0.5), range = c(0.01, 3), step = 0.01, lty = 6)
p <- tda_pl_text(p, "b = 0.0", at = c(2.6, 1.1))
p <- tda_pl_text(p, "b = 1.2", at = c(2.6, 1.6))
p <- tda_pl_text(p, "b = 1.5", at = c(2.6, 2.2))
p <- tda_pl_text(p, "b = 0.7", at = c(2.6, 0.6))
p <- tda_pl_text(p, "b = 0.5", at = c(2.6, 0.1))
plot(p)
6.17.3.4 Sickle Models
Box 148Sickle (rt8.cf)
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 8)Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 8)
Episodes: 600
Model: 8 (TDA 8)
logLik (starting values): -2897.762
logLik: -2466.1
Converged in 7 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Constant -7.0645 0.1140 -61.9618 1.0000
2 1 0 1 A COHO2 0.3926 0.1123 3.4947 0.9995
3 1 0 1 A COHO3 0.4363 0.1162 3.7556 0.9998
4 1 0 1 A W 0.4232 0.0944 4.4808 1.0000
5 1 0 1 B Constant 3.7046 0.0517 71.6511 1.0000Box 149plot-sic.cf
sic <- function(b) sprintf("1*x1*exp(-x1/%s)", b)
p <- tda_ps(xlim = c(0, 12), ylim = c(0, 0.8), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(1, 0.4), ic = c(2, 4))
p <- tda_pl_frame(p)
p <- tda_pl_function(p, sic(1.0), range = c(0, 12), step = 0.1)
p <- tda_pl_function(p, sic(0.5), range = c(0, 12), step = 0.1, lty = 5)
p <- tda_pl_function(p, sic(2.0), range = c(0, 12), step = 0.1, lty = 6)
p <- tda_pl_text(p, "b = 1.0", at = c(4, 0.11))
p <- tda_pl_text(p, "b = 0.5", at = c(2, 0.06))
p <- tda_pl_text(p, "b = 2.0", at = c(10, 0.14))
plot(p)
6.17.3.5 Log-logistic Models
Box 150Log-logistic (rt9.cf)
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 9)Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 9)
Episodes: 600
Model: 9 (TDA 9)
logLik (starting values): -2521.785
logLik: -2442.302
Converged in 5 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Constant -4.3546 0.0998 -43.6183 1.0000
2 1 0 1 A COHO2 0.4792 0.1257 3.8138 0.9999
3 1 0 1 A COHO3 0.5927 0.1295 4.5752 1.0000
4 1 0 1 A W 0.3989 0.1065 3.7463 0.9998
5 1 0 1 B Constant 0.3307 0.0385 8.5823 1.0000Box 151plot-ll.cf
ll <- function(b) sprintf("%s*x1^(%s-1)/(1+x1^%s)", b, b, b)
p <- tda_ps(xlim = c(0, 3), ylim = c(0, 2), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(1, 1))
p <- tda_pl_frame(p)
p <- tda_pl_function(p, ll(1.0), range = c(0.01, 3), step = 0.02)
p <- tda_pl_function(p, ll(1.5), range = c(0.01, 3), step = 0.02, lty = 5)
p <- tda_pl_function(p, ll(2.0), range = c(0.01, 3), step = 0.02, lty = 6)
p <- tda_pl_function(p, ll(0.7), range = c(0.01, 3), step = 0.02, lty = 5)
p <- tda_pl_function(p, ll(0.4), range = c(0.01, 3), step = 0.02, lty = 6)
p <- tda_pl_text(p, "0.4", at = c(2.8, 0.016), fs = 1.5)
p <- tda_pl_text(p, "0.7", at = c(2.8, 0.1), fs = 1.5)
p <- tda_pl_text(p, "1.0", at = c(2.8, 0.18), fs = 1.5)
p <- tda_pl_text(p, "1.5", at = c(2.8, 0.33), fs = 1.5)
p <- tda_pl_text(p, "2.0", at = c(2.8, 0.52), fs = 1.5)
plot(p)
6.17.3.6 Log-normal Models
Box 152Log-normal (rt12.cf)
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 12)Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 12)
Episodes: 600
Model: 12 (TDA 12)
logLik (starting values): -2603.168
logLik: -2437.524
Converged in 6 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Constant 4.3688 0.0960 45.5055 1.0000
2 1 0 1 A COHO2 -0.4608 0.1255 -3.6707 0.9998
3 1 0 1 A COHO3 -0.6110 0.1272 -4.8036 1.0000
4 1 0 1 A W -0.3821 0.1062 -3.5992 0.9997
5 1 0 1 B Constant 0.2036 0.0342 5.9508 1.0000Box 153plot-ln.cf
ln <- function(b) sprintf("ndf(log(x1)/%s)/(%s*x1*(1-nd(log(x1)/%s)))",
b, b, b)
p <- tda_ps(xlim = c(0, 7), ylim = c(0, 1.5), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(1, 0.5))
p <- tda_pl_frame(p)
p <- tda_pl_function(p, ln(1.0), range = c(0.02, 7), step = 0.02)
p <- tda_pl_function(p, ln(0.7), range = c(0.02, 7), step = 0.02, lty = 5)
p <- tda_pl_function(p, ln(1.3), range = c(0.02, 7), step = 0.02, lty = 6)
p <- tda_pl_text(p, "b = 1.3", at = c(6, 0.125))
p <- tda_pl_text(p, "b = 1.0", at = c(6, 0.4))
p <- tda_pl_text(p, "b = 0.7", at = c(6, 0.725))
plot(p)
6.17.3.7 Generalized Gamma Models
Box 154Generalized gamma (rt13.cf)
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 13)Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 13)
Episodes: 600
Model: 13 (TDA 13)
logLik (starting values): -2514.02
logLik: -2473.543
Converged in 6 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Constant 5.0055 0.0906 55.2452 1.0000
2 1 0 1 A COHO2 -0.5339 0.1201 -4.4448 1.0000
3 1 0 1 A COHO3 -0.6671 0.1235 -5.4003 1.0000
4 1 0 1 A W -0.5191 0.1012 -5.1274 1.0000
5 1 0 1 B Constant 0.0700 0.0366 1.9119 0.9441The generalised gamma rate at three values of the shape parameter,
written out with TDA’s lgam() and icg(), the
log gamma and the incomplete gamma function.
Box 155plot-g1.cf
gg <- function(k) sprintf(
"%s^(%s-0.5)*exp(sqrt(%s)*log(x1)-%s*exp(log(x1)/sqrt(%s)))/(x1*exp(lgam(%s))*(1-icg(%s*exp(log(x1)/sqrt(%s)),%s)))",
k, k, k, k, k, k, k, k, k)
p <- tda_ps(xlim = c(0, 5), ylim = c(0, 2), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(1, 1))
p <- tda_pl_frame(p)
p <- tda_pl_function(p, gg(1), range = c(0.01, 5), step = 0.02)
p <- tda_pl_function(p, gg(0.5), range = c(0.01, 5), step = 0.02, lty = 5)
p <- tda_pl_function(p, gg(10), range = c(0.01, 5), step = 0.02, lty = 6)
p <- tda_pl_text(p, "a = 0.0", at = c(0.5, 1.73))
p <- tda_pl_text(p, "b = 1.0", at = c(0.5, 1.57))
p <- tda_pl_text(p, "k = 10.0", at = c(4, 0.63))
p <- tda_pl_text(p, "k = 1.0", at = c(4, 1.07))
p <- tda_pl_text(p, "k = 0.5", at = c(4, 1.5))
plot(p)
Box 156plot-g2.cf
gg2 <- function(b) sprintf(
"1^(1-0.5)*exp(sqrt(1)*log(x1)/%s-1*exp((log(x1)/%s)/sqrt(1)))/(x1*exp(lgam(1))*(1-icg(1*exp((log(x1)/%s)/sqrt(1)),1)))",
b, b, b)
p <- tda_ps(xlim = c(0, 5), ylim = c(0, 3), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(1, 1))
p <- tda_pl_frame(p)
p <- tda_pl_function(p, gg2(1), range = c(0.01, 5), step = 0.02)
p <- tda_pl_function(p, gg2(0.7), range = c(0.01, 5), step = 0.02, lty = 5)
p <- tda_pl_function(p, gg2(2), range = c(0.01, 5), step = 0.02, lty = 6)
p <- tda_pl_text(p, "a = 0.0", at = c(0.5, 2.6))
p <- tda_pl_text(p, "k = 1.0", at = c(0.5, 2.35))
p <- tda_pl_text(p, "b = 2.0", at = c(4, 0.6))
p <- tda_pl_text(p, "b = 1.0", at = c(4, 1.1))
p <- tda_pl_text(p, "b = 0.7", at = c(4, 2))
plot(p)
6.17.3.8 Inverse Gaussian Models
Box 157Inverse Gaussian (rt14.cf)
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 14)Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 14)
Episodes: 600
Model: 14 (TDA 14)
logLik (starting values): -2699.618
logLik: -2434.345
Converged in 6 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Constant 0.0007 0.0015 0.4225 0.3273
2 1 0 1 A COHO2 0.0067 0.0024 2.8243 0.9953
3 1 0 1 A COHO3 0.0088 0.0026 3.4284 0.9994
4 1 0 1 A W 0.0093 0.0021 4.4894 1.0000
5 1 0 1 B Constant -1.7092 0.0311 -55.0104 1.0000The inverse Gaussian rate, written out with TDA’s ndf()
and nd().
Box 158plot-ig.cf
ig <- function(a) sprintf(
"(ndf((%s*x1-1)/sqrt(x1))/x1^(3/2))/(nd((1-%s*x1)/sqrt(x1))-exp(2*%s)*nd((-1-%s*x1)/sqrt(x1)))",
a, a, a, a)
p <- tda_ps(xlim = c(0, 6), ylim = c(0, 2.5), width = 90, height = 50)
p <- tda_pl_axes(p, sc = c(1, 0.5))
p <- tda_pl_frame(p)
p <- tda_pl_function(p, ig(1), range = c(0.02, 6), step = 0.02)
p <- tda_pl_function(p, ig(0.1), range = c(0.02, 6), step = 0.02, lty = 5)
p <- tda_pl_function(p, ig(1.5), range = c(0.02, 6), step = 0.02, lty = 6)
p <- tda_pl_text(p, "a = 0.1", at = c(5, 0.21))
p <- tda_pl_text(p, "a = 1.0", at = c(5, 0.83))
p <- tda_pl_text(p, "a = 1.5", at = c(5, 1.46))
p <- tda_pl_text(p, "b = 1.0", at = c(5, 2.08))
plot(p)
6.17.5.1 The frml Command
tda_frml() is the rate-model counterpart of
tda_fml(): the episode is named by the formula, and the
likelihood is written out. tf, ts and
des are available inside the block, which is what makes a
model with its duration dependence writable.
Example 1 The exponential model again, now by hand.
Box 159An exponential model as frml (fml1.cf)
tda_frml(Surv(TFP, DES) ~ COHO2 + COHO3 + W, {
rate = exp(a0 + COHO2 * a1 + COHO3 * a2 + W * a3)
l1 = ifelse(DES, log(rate), 0)
fn = l1 - rate * TFP
}, data = d, start = c(-4, 0, 0, 0))Call: tda_frml(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
definitions = {
rate = exp(a0 + COHO2 * a1 + COHO3 * a2 + W * a3)
l1 = ifelse(DES, log(rate), 0)
fn = l1 - rate * TFP
}, start = c(-4, 0, 0, 0))
Episodes: 600
logLik (starting values): -2578.948
logLik: -2475.438
Converged in 6 iterations
Idx Parameter Coeff Error C/Error Signif
1 a0 -5.0114 0.0843 -59.4446 1.0000
2 a1 0.5341 0.1120 4.7686 1.0000
3 a2 0.6738 0.1152 5.8472 1.0000
4 a3 0.5065 0.0942 5.3746 1.0000Example 2 frml1m.cf: three competing
destinations, each with its own rate, the episode contributing the log
of whichever rate it realised and the sum of all three to the survivor
term. Its DES is rt1m.cf’s (the
ge coding).
Box 160Three competing rates (frml1m.cf)
tda_frml(Surv(TFP, DES) ~ COHO2 + COHO3 + W, {
rate1 = exp(a10 + COHO2 * a11 + COHO3 * a12 + W * a13)
rate2 = exp(a20 + COHO2 * a21 + COHO3 * a22 + W * a23)
rate3 = exp(a30 + COHO2 * a31 + COHO3 * a32 + W * a33)
rate = rate1 + rate2 + rate3
l1 = ifelse(DES == 1, log(rate1),
ifelse(DES == 2, log(rate2),
ifelse(DES == 3, log(rate3), 0)))
fn = l1 - rate * TFP
}, data = d4m, start = rep(c(-4, 0, 0, 0), 3),
control = tda_control(maxit = 100))Call: tda_frml(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d4m,
definitions = {
rate1 = exp(a10 + COHO2 * a11 + COHO3 * a12 + W * a13)
rate2 = exp(a20 + COHO2 * a21 + COHO3 * a22 + W * a23)
rate3 = exp(a30 + COHO2 * a31 + COHO3 * a32 + W * a33)
rate = rate1 + rate2 + rate3
l1 = ifelse(DES == 1, log(rate1), ifelse(DES == 2, log(rate2),
ifelse(DES == 3, log(rate3), 0)))
fn = l1 - rate * TFP
}, start = rep(c(-4, 0, 0, 0), 3), control = tda_control(maxit = 100))
Episodes: 600
logLik (starting values): -4072.845
logLik: -2935.583
Converged in 7 iterations
Idx Parameter Coeff Error C/Error Signif
1 a10 -6.3412 0.1736 -36.5229 1.0000
2 a11 0.3012 0.2618 1.1504 0.7500
3 a12 0.4999 0.2651 1.8857 0.9407
4 a13 -0.0957 0.2342 -0.4084 0.3170
5 a20 -5.7548 0.1236 -46.5439 1.0000
6 a21 0.6924 0.1630 4.2486 1.0000
7 a22 0.8088 0.1684 4.8039 1.0000
8 a23 0.2977 0.1381 2.1556 0.9689
9 a30 -6.3686 0.1578 -40.3698 1.0000
10 a31 0.4424 0.1924 2.2996 0.9785
11 a32 0.5875 0.1980 2.9676 0.9970
12 a33 1.1156 0.1659 6.7254 1.0000Example 2a frml2.cf writes the
piecewise constant model of rt2.cf out by hand: one
indicator per period, and the log rate as the sum of the period
constants and the covariate effects; frml2c.cf adds the
constraints that equate the periods. Both reach rt2 and
rt2c.
Box 161frml2.cf, frml2c.cf: the piecewise constant model, by hand
pw <- function(...) tda_frml(Surv(TFP, DES) ~ COHO2 + COHO3 + W, {
bb = COHO2 * b1 + COHO3 * b2 + W * b3
p1 = ge(tf, 0) & lt(tf, 12); p2 = ge(tf, 12) & lt(tf, 24)
p3 = ge(tf, 24) & lt(tf, 36); p4 = ge(tf, 36) & lt(tf, 48)
p5 = ge(tf, 48) & lt(tf, 60); p6 = ge(tf, 60) & lt(tf, 72)
p7 = ge(tf, 72) & lt(tf, 84); p8 = ge(tf, 84) & lt(tf, 96)
p9 = ge(tf, 96)
rr = ifelse(p1, a1, ifelse(p2, a2, ifelse(p3, a3, ifelse(p4, a4,
ifelse(p5, a5, ifelse(p6, a6, ifelse(p7, a7, ifelse(p8, a8,
ifelse(p9, a9, 0)))))))))
s1 = 12 * exp(a1 + bb); s2 = s1 + 12 * exp(a2 + bb)
s3 = s2 + 12 * exp(a3 + bb); s4 = s3 + 12 * exp(a4 + bb)
s5 = s4 + 12 * exp(a5 + bb); s6 = s5 + 12 * exp(a6 + bb)
s7 = s6 + 12 * exp(a7 + bb); s8 = s7 + 12 * exp(a8 + bb)
lsurv = ifelse(p1, tf * exp(a1 + bb),
ifelse(p2, s1 + (tf - 12) * exp(a2 + bb),
ifelse(p3, s2 + (tf - 24) * exp(a3 + bb),
ifelse(p4, s3 + (tf - 36) * exp(a4 + bb),
ifelse(p5, s4 + (tf - 48) * exp(a5 + bb),
ifelse(p6, s5 + (tf - 60) * exp(a6 + bb),
ifelse(p7, s6 + (tf - 72) * exp(a7 + bb),
ifelse(p8, s7 + (tf - 84) * exp(a8 + bb),
ifelse(p9, s8 + (tf - 96) * exp(a9 + bb), 0)))))))))
l1 = ifelse(DES, rr + bb, 0)
fn = l1 - lsurv
}, data = d, start = c(rep(-4, 9), 0, 0, 0), ...)
pw() # frml2.cfCall: tda_frml(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
definitions = {
bb = COHO2 * b1 + COHO3 * b2 + W * b3
p1 = ge(tf, 0) & lt(tf, 12)
p2 = ge(tf, 12) & lt(tf, 24)
p3 = ge(tf, 24) & lt(tf, 36)
p4 = ge(tf, 36) & lt(tf, 48)
p5 = ge(tf, 48) & lt(tf, 60)
p6 = ge(tf, 60) & lt(tf, 72)
p7 = ge(tf, 72) & lt(tf, 84)
p8 = ge(tf, 84) & lt(tf, 96)
p9 = ge(tf, 96)
rr = ifelse(p1, a1, ifelse(p2, a2, ifelse(p3, a3, ifelse(p4,
a4, ifelse(p5, a5, ifelse(p6, a6, ifelse(p7, a7,
ifelse(p8, a8, ifelse(p9, a9, 0)))))))))
s1 = 12 * exp(a1 + bb)
s2 = s1 + 12 * exp(a2 + bb)
s3 = s2 + 12 * exp(a3 + bb)
s4 = s3 + 12 * exp(a4 + bb)
s5 = s4 + 12 * exp(a5 + bb)
s6 = s5 + 12 * exp(a6 + bb)
s7 = s6 + 12 * exp(a7 + bb)
s8 = s7 + 12 * exp(a8 + bb)
lsurv = ifelse(p1, tf * exp(a1 + bb), ifelse(p2, s1 +
(tf - 12) * exp(a2 + bb), ifelse(p3, s2 + (tf - 24) *
exp(a3 + bb), ifelse(p4, s3 + (tf - 36) * exp(a4 +
bb), ifelse(p5, s4 + (tf - 48) * exp(a5 + bb), ifelse(p6,
s5 + (tf - 60) * exp(a6 + bb), ifelse(p7, s6 + (tf -
72) * exp(a7 + bb), ifelse(p8, s7 + (tf - 84) *
exp(a8 + bb), ifelse(p9, s8 + (tf - 96) * exp(a9 +
bb), 0)))))))))
l1 = ifelse(DES, rr + bb, 0)
fn = l1 - lsurv
}, start = c(rep(-4, 9), 0, 0, 0))
Episodes: 600
logLik (starting values): -2578.948
logLik: -2433.376
Converged in 7 iterations
Idx Parameter Coeff Error C/Error Signif
1 a1 -5.2070 0.1520 -34.2464 1.0000
2 a2 -4.5473 0.1312 -34.6574 1.0000
3 a3 -4.2675 0.1296 -32.9389 1.0000
4 a4 -4.7784 0.1690 -28.2758 1.0000
5 a5 -4.7300 0.1779 -26.5924 1.0000
6 a6 -4.7862 0.1987 -24.0857 1.0000
7 a7 -5.1366 0.2523 -20.3568 1.0000
8 a8 -6.0783 0.4137 -14.6917 1.0000
9 a9 -5.4198 0.1281 -42.3123 1.0000
10 b1 0.4367 0.1133 3.8534 0.9999
11 b2 0.5162 0.1182 4.3688 1.0000
12 b3 0.4336 0.0949 4.5687 1.0000pw(constraints = sprintf("b%d - b%d = 0", 2:9, 1:8)) # frml2c.cfCall: tda_frml(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
definitions = {
bb = COHO2 * b1 + COHO3 * b2 + W * b3
p1 = ge(tf, 0) & lt(tf, 12)
p2 = ge(tf, 12) & lt(tf, 24)
p3 = ge(tf, 24) & lt(tf, 36)
p4 = ge(tf, 36) & lt(tf, 48)
p5 = ge(tf, 48) & lt(tf, 60)
p6 = ge(tf, 60) & lt(tf, 72)
p7 = ge(tf, 72) & lt(tf, 84)
p8 = ge(tf, 84) & lt(tf, 96)
p9 = ge(tf, 96)
rr = ifelse(p1, a1, ifelse(p2, a2, ifelse(p3, a3, ifelse(p4,
a4, ifelse(p5, a5, ifelse(p6, a6, ifelse(p7, a7,
ifelse(p8, a8, ifelse(p9, a9, 0)))))))))
s1 = 12 * exp(a1 + bb)
s2 = s1 + 12 * exp(a2 + bb)
s3 = s2 + 12 * exp(a3 + bb)
s4 = s3 + 12 * exp(a4 + bb)
s5 = s4 + 12 * exp(a5 + bb)
s6 = s5 + 12 * exp(a6 + bb)
s7 = s6 + 12 * exp(a7 + bb)
s8 = s7 + 12 * exp(a8 + bb)
lsurv = ifelse(p1, tf * exp(a1 + bb), ifelse(p2, s1 +
(tf - 12) * exp(a2 + bb), ifelse(p3, s2 + (tf - 24) *
exp(a3 + bb), ifelse(p4, s3 + (tf - 36) * exp(a4 +
bb), ifelse(p5, s4 + (tf - 48) * exp(a5 + bb), ifelse(p6,
s5 + (tf - 60) * exp(a6 + bb), ifelse(p7, s6 + (tf -
72) * exp(a7 + bb), ifelse(p8, s7 + (tf - 84) *
exp(a8 + bb), ifelse(p9, s8 + (tf - 96) * exp(a9 +
bb), 0)))))))))
l1 = ifelse(DES, rr + bb, 0)
fn = l1 - lsurv
}, start = c(rep(-4, 9), 0, 0, 0), constraints = ..1)
Episodes: 600
logLik (starting values): -2578.948
logLik: -2475.438
Converged in 6 iterations
Idx Parameter Coeff Error C/Error Signif
1 a1 -5.0114 0.0843 -59.4446 1.0000
2 a2 -5.0114 0.0843 -59.4446 1.0000
3 a3 -5.0114 0.0843 -59.4446 1.0000
4 a4 -5.0114 0.0843 -59.4446 1.0000
5 a5 -5.0114 0.0843 -59.4446 1.0000
6 a6 -5.0114 0.0843 -59.4446 1.0000
7 a7 -5.0114 0.0843 -59.4446 1.0000
8 a8 -5.0114 0.0843 -59.4446 1.0000
9 a9 -5.0114 0.0843 -59.4446 1.0000
10 b1 0.5341 0.1120 4.7686 1.0000
11 b2 0.6738 0.1152 5.8472 1.0000
12 b3 0.5065 0.0942 5.3746 1.0000Example 3 A Weibull and a log-normal, written out.
The second uses nd and ndf, TDA’s normal
distribution and density operators.
Box 162Weibull and log-normal, written out
tda_frml(Surv(TFP, DES) ~ COHO2 + COHO3 + W, {
aa = exp(a0 + COHO2 * a1 + COHO3 * a2 + W * a3)
bb = exp(b0)
l1 = ifelse(DES, log(bb * aa^bb * tf^(bb - 1)), 0)
fn = l1 - (aa * tf)^bb
}, data = d, start = c(-4, 0, 0, 0, 0), control = tda_control(maxit = 100))Call: tda_frml(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
definitions = {
aa = exp(a0 + COHO2 * a1 + COHO3 * a2 + W * a3)
bb = exp(b0)
l1 = ifelse(DES, log(bb * aa^bb * tf^(bb - 1)), 0)
fn = l1 - (aa * tf)^bb
}, start = c(-4, 0, 0, 0, 0), control = tda_control(maxit = 100))
Episodes: 600
logLik (starting values): -2578.948
logLik: -2473.543
Converged in 6 iterations
Idx Parameter Coeff Error C/Error Signif
1 a0 -5.0055 0.0906 -55.2452 1.0000
2 a1 0.5339 0.1201 4.4448 1.0000
3 a2 0.6671 0.1235 5.4003 1.0000
4 a3 0.5191 0.1012 5.1274 1.0000
5 b0 -0.0700 0.0366 -1.9119 0.9441tda_frml(Surv(TFP, DES) ~ COHO2 + COHO3 + W, {
aa = a0 + COHO2 * a1 + COHO3 * a2 + W * a3
bb = exp(b0)
zz = (log(tf) - aa) / bb
qz = 1 - nd(zz)
vz = ndf(zz) / qz
l1 = ifelse(DES, log(vz / (bb * tf)), 0)
fn = l1 + log(qz)
}, data = d, start = c(4, 0, 0, 0, 0), control = tda_control(maxit = 100))Call: tda_frml(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
definitions = {
aa = a0 + COHO2 * a1 + COHO3 * a2 + W * a3
bb = exp(b0)
zz = (log(tf) - aa)/bb
qz = 1 - nd(zz)
vz = ndf(zz)/qz
l1 = ifelse(DES, log(vz/(bb * tf)), 0)
fn = l1 + log(qz)
}, start = c(4, 0, 0, 0, 0), control = tda_control(maxit = 100))
Episodes: 600
logLik (starting values): -2493.477
logLik: -2437.524
Converged in 6 iterations
Idx Parameter Coeff Error C/Error Signif
1 a0 4.3688 0.0960 45.5055 1.0000
2 a1 -0.4608 0.1255 -3.6707 0.9998
3 a2 -0.6110 0.1272 -4.8035 1.0000
4 a3 -0.3821 0.1062 -3.5992 0.9997
5 b0 0.2036 0.0342 5.9508 1.0000Example 4 frml5.cf is the generalized
gamma model written out, with icg, the incomplete gamma
integral, and lgam; with the shape kk = 1 it
is the Weibull of Example 3, which is what the box shows.
Box 163frml5.cf: the generalized gamma, written out
tda_frml(Surv(TFP, DES) ~ COHO2 + COHO3 + W, {
kk = 1
aa = a0 + COHO2 * a1 + COHO3 * a2 + W * a3
bb = exp(b0)
zz = (log(tf) - aa) / bb
qt = kk * exp(zz / sqrt(kk))
rr = (kk - 0.5) * log(kk) + (sqrt(kk) * zz - qt) - (b0 + log(tf) + lgam(kk))
icc = icg(qt, kk)
fn = ifelse(DES, rr, log(1 - icc))
}, data = d, start = c(5, 0, 0, 0, 0), control = tda_control(algorithm = 4))Call: tda_frml(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
definitions = {
kk = 1
aa = a0 + COHO2 * a1 + COHO3 * a2 + W * a3
bb = exp(b0)
zz = (log(tf) - aa)/bb
qt = kk * exp(zz/sqrt(kk))
rr = (kk - 0.5) * log(kk) + (sqrt(kk) * zz - qt) - (b0 +
log(tf) + lgam(kk))
icc = icg(qt, kk)
fn = ifelse(DES, rr, log(1 - icc))
}, start = c(5, 0, 0, 0, 0), control = tda_control(algorithm = 4))
Episodes: 600
logLik (starting values): -2564.787
logLik: -2473.543
Converged in 20 iterations
Idx Parameter Coeff Error C/Error Signif
1 a0 5.0055 0.0906 55.2452 1.0000
2 a1 -0.5339 0.1201 -4.4448 1.0000
3 a2 -0.6671 0.1235 -5.4003 1.0000
4 a3 -0.5191 0.1012 -5.1274 1.0000
5 b0 0.0700 0.0366 1.9119 0.94416.17.5.2 The Hernes Model
Box 164The Hernes model (frml8.cf)
tda_frml(Surv(TFP, DES) ~ COHO2 + COHO3 + W, {
sigma = exp(s)
lambda = exp(l) / (1 + exp(l))
beta = exp(b)
ff = sigma * exp(beta * lambda^tf)
ff1 = ff + 1
dens = -(ff * beta * lambda^tf * log(lambda)) / (ff1 * ff1)
surv = 1 - 1 / ff1
fn = ifelse(des, log(dens), log(surv))
}, data = d, control = tda_control(maxit = 100, algorithm = 4))Call: tda_frml(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
definitions = {
sigma = exp(s)
lambda = exp(l)/(1 + exp(l))
beta = exp(b)
ff = sigma * exp(beta * lambda^tf)
ff1 = ff + 1
dens = -(ff * beta * lambda^tf * log(lambda))/(ff1 *
ff1)
surv = 1 - 1/ff1
fn = ifelse(des, log(dens), log(surv))
}, control = tda_control(maxit = 100, algorithm = 4))
Episodes: 600
logLik (starting values): -16552.96
logLik: -2526.374
Converged in 22 iterations
Idx Parameter Coeff Error C/Error Signif
1 b 1.5856 0.0378 41.9814 1.0000
2 l 3.8236 0.0599 63.8607 1.0000
3 s -1.8379 0.1305 -14.0821 1.00006.17.5.3 The Coale-McNeil Model
The manual’s frml9.cf gives two starting values for
three parameters and lets TDA pad the third with zero, which TDA warns
about; the same values, spelled out, reach the same estimates without
the warning.
Box 165The Coale-McNeil model (frml9.cf)
tda_frml(Surv(TFP, DES) ~ COHO2 + COHO3 + W, {
bg = 0.604
sigma = exp(s) / (1 + exp(s))
lambda = exp(l)
mue = exp(m)
beta = bg * lambda
tl = exp(-lambda * (tf - mue))
ldens = log(sigma) + l - beta * (tf - mue) - tl - lgam(bg)
surv = 1 - sigma * (1 - icg(tl, bg))
fn = ifelse(DES, ldens, log(surv))
}, data = d, start = c(l = -3, m = 2, s = 0),
control = tda_control(maxit = 100, algorithm = 4))Call: tda_frml(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
definitions = {
bg = 0.604
sigma = exp(s)/(1 + exp(s))
lambda = exp(l)
mue = exp(m)
beta = bg * lambda
tl = exp(-lambda * (tf - mue))
ldens = log(sigma) + l - beta * (tf - mue) - tl - lgam(bg)
surv = 1 - sigma * (1 - icg(tl, bg))
fn = ifelse(DES, ldens, log(surv))
}, start = c(l = -3, m = 2, s = 0), control = tda_control(maxit = 100,
algorithm = 4))
Episodes: 600
logLik (starting values): -2699.607
logLik: -2529.593
Converged in 13 iterations
Idx Parameter Coeff Error C/Error Signif
1 l -3.0653 0.0454 -67.4629 1.0000
2 m 2.8902 0.0685 42.1764 1.0000
3 s 1.8505 0.1421 13.0217 1.00006.17.6.2 Logistic Regression Models
Model 20: the discrete-time logistic model, the rate at each time point a logit in the covariates.
Box 166Discrete time logistic
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 20)Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 20)
Episodes: 600
Model: 20 (TDA 20)
logLik (starting values): -2511.467
logLik: -2472.389
Converged in 5 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Constant -5.0056 0.0847 -59.1117 1.0000
2 1 0 1 A COHO2 0.5402 0.1127 4.7944 1.0000
3 1 0 1 A COHO3 0.6822 0.1160 5.8808 1.0000
4 1 0 1 A W 0.5133 0.0949 5.4097 1.00006.17.6.3 Complementary Log-Log Models
Model 21.
Box 167Complementary log-log
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 21)Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 21)
Episodes: 600
Model: 21 (TDA 21)
logLik (starting values): -2511.446
logLik: -2472.421
Converged in 5 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A Constant -5.0081 0.0843 -59.4116 1.0000
2 1 0 1 A COHO2 0.5371 0.1120 4.7959 1.0000
3 1 0 1 A COHO3 0.6773 0.1152 5.8768 1.0000
4 1 0 1 A W 0.5094 0.0942 5.4044 1.00006.17.7.1 Partial Likelihood Estimation
Model 1 is the Cox model, estimated by partial likelihood, here with one transition and then with three.
Box 168pl1.cf and pl1m.cf: Cox models with one and three transitions
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 1)Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d,
model = 1)
Episodes: 600
Model: 1 (TDA 1)
logLik (starting values): -2584.57
logLik: -2565.426
Converged in 4 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A COHO2 0.3797 0.1134 3.3501 0.9992
2 1 0 1 A COHO3 0.4585 0.1177 3.8937 0.9999
3 1 0 1 A W 0.4078 0.0949 4.2988 1.0000tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d4m, model = 1)Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d4m,
model = 1)
Episodes: 600
Model: 1 (TDA 1)
logLik (starting values): -2584.57
logLik: -2550.89
Converged in 4 iterations
Transitions: 0->1, 0->2, 0->3
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A COHO2 0.0893 0.2640 0.3381 0.2647
2 1 0 1 A COHO3 0.1624 0.2696 0.6024 0.4531
3 1 0 1 A W -0.2411 0.2352 -1.0254 0.6948
4 1 0 2 A COHO2 0.5017 0.1641 3.0564 0.9978
5 1 0 2 A COHO3 0.5487 0.1714 3.2010 0.9986
6 1 0 2 A W 0.1879 0.1387 1.3547 0.8245
7 1 0 3 A COHO2 0.3726 0.1964 1.8966 0.9421
8 1 0 3 A COHO3 0.4909 0.2039 2.4069 0.9839
9 1 0 3 A W 1.0651 0.1678 6.3482 1.00006.17.7.2 Time-dependent Covariates
define= writes a covariate as a TDA expression in
time, evaluated at every risk set rather than once per
episode, and helpers= names the columns that expression
needs. MDATE does not depend on time and so is
computed in R; MARR does, and so is defined for TDA.
Box 169A time-dependent covariate
d$MDATE <- ifelse(d$TMAR <= 0, 10000, d$TMAR - d$TStart)
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W + MARR, data = d, model = 1,
define = list(MARR = "gt(time,MDATE)"), helpers = "MDATE")Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W + MARR,
data = d, model = 1, define = list(MARR = "gt(time,MDATE)"),
helpers = "MDATE")
Episodes: 600
Model: 1 (TDA 1)
logLik (starting values): -2584.57
logLik: -2562.509
Converged in 4 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A COHO2 0.3920 0.1135 3.4548 0.9994
2 1 0 1 A COHO3 0.4668 0.1179 3.9604 0.9999
3 1 0 1 A W 0.3947 0.0950 4.1543 1.0000
4 1 0 1 A MARR -0.2408 0.1000 -2.4083 0.98406.17.7.3 Episode Splitting
pl4.cf splits every episode at the date of marriage
(split = MDATE in edef) and writes the pieces
out; pl5.cf reads them back and fits the Cox model twice,
without and with MARR, the piece’s marriage status.
tda_split(at=) is the split, and the pieces are left
truncated, so the model is told Surv(ts, tf, des).
Box 170pl4.cf, pl5.cf: episode splitting
d$MDATE <- ifelse(d$TMAR <= 0, 10000, d$TMAR - d$TStart)
sp <- tda_split(Surv(TFP, DES) ~ COHO2 + COHO3 + W, d, at = "MDATE")
sp$MARR <- as.integer(sp$MDATE <= sp$ts)
nrow(sp)[1] 761tda_rate(Surv(ts, tf, des) ~ COHO2 + COHO3 + W, sp, model = 1)Call: tda_rate(formula = Surv(ts, tf, des) ~ COHO2 + COHO3 + W, data = sp,
model = 1)
Episodes: 761
Model: 1 (TDA 1)
logLik (starting values): -2584.57
logLik: -2565.426
Converged in 4 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A COHO2 0.3797 0.1134 3.3501 0.9992
2 1 0 1 A COHO3 0.4585 0.1177 3.8937 0.9999
3 1 0 1 A W 0.4078 0.0949 4.2988 1.0000tda_rate(Surv(ts, tf, des) ~ COHO2 + COHO3 + W + MARR, sp, model = 1)Call: tda_rate(formula = Surv(ts, tf, des) ~ COHO2 + COHO3 + W + MARR,
data = sp, model = 1)
Episodes: 761
Model: 1 (TDA 1)
logLik (starting values): -2584.57
logLik: -2562.509
Converged in 4 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A COHO2 0.3920 0.1135 3.4548 0.9994
2 1 0 1 A COHO3 0.4668 0.1179 3.9604 0.9999
3 1 0 1 A W 0.3947 0.0950 4.1543 1.0000
4 1 0 1 A MARR -0.2408 0.1000 -2.4083 0.98406.17.7.4 The Proportionality Assumption
The usual test is an interaction with log time: if the effect is proportional, its coefficient is zero.
Box 171A test of proportionality (pl6.cf)
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W + WTEST, data = d, model = 1,
define = list(WTEST = "W * (log(time) - 4.22)"))Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + W + WTEST,
data = d, model = 1, define = list(WTEST = "W * (log(time) - 4.22)"))
Episodes: 600
Model: 1 (TDA 1)
logLik (starting values): -2584.57
logLik: -2562.903
Converged in 4 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A COHO2 0.3816 0.1133 3.3682 0.9992
2 1 0 1 A COHO3 0.4381 0.1180 3.7136 0.9998
3 1 0 1 A W 0.5840 0.1215 4.8074 1.0000
4 1 0 1 A WTEST 0.2196 0.0974 2.2541 0.9758The same question asked by splitting the time axis: tp=
estimates within periods.
Box 172The effect within periods (pl7.cf)
rrdat5 <- read.table(ex("rrdat.5"), col.names = c("TF", "DES", "G"))
rrdat5$G1 <- as.integer(rrdat5$G == 1)
pl7 <- tda_rate(Surv(TF, DES) ~ G1, data = rrdat5, model = 1,
tp = c(0, 170.1, 354.1, 535.1))
pl7Call: tda_rate(formula = Surv(TF, DES) ~ G1, data = rrdat5, model = 1,
tp = c(0, 170.1, 354.1, 535.1))
Episodes: 90
Model: 1 (TDA 1)
logLik (starting values): -283.3952
logLik: -282.7441
Converged in 3 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A G1 -0.2666 0.2332 -1.1428 0.7469pl7$gof sn org des statistic df significance
1 1 0 1 10.23511 3 0.9833302$gof holds the global goodness-of-fit statistic that
comes with a period split – the manual’s Box 5, from pl8.cf
for the single transition and pl8m.cf for the three.
Box 173pl8.cf, pl8m.cf: goodness of fit over periods
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 1,
tp = seq(0, 96, 12))$gof sn org des statistic df significance
1 1 0 1 37.10844 24 0.9573181tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d4m, model = 1,
tp = seq(0, 96, 12))$gof sn org des statistic df significance
1 1 0 1 15.90665 24 0.1085840
2 1 0 2 25.54543 24 0.6233751
3 1 0 3 38.57908 24 0.96978266.17.7.5 Stratification
strata() on the right hand side is survival’s spelling
and TDA’s grp=: a separate baseline rate per stratum, one
set of coefficients.
Box 174A stratified Cox model (pl9.cf)
tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + strata(SEX), data = d, model = 1)Call: tda_rate(formula = Surv(TFP, DES) ~ COHO2 + COHO3 + strata(SEX),
data = d, model = 1)
Episodes: 600
Model: 1 (TDA 1)
logLik (starting values): -2259.567
logLik: -2250.583
Converged in 4 iterations
Idx SN Org Des MT Variable Coeff Error C/Error Signif
1 1 0 1 A COHO2 0.3840 0.1133 3.3878 0.9993
2 1 0 1 A COHO3 0.4428 0.1183 3.7433 0.99986.17.7.6 The Baseline Rate
prate= evaluates the baseline rate at a covariate
vector.
Box 175The baseline rate at given covariate values (pl1r.cf)
fit <- tda_rate(Surv(TFP, DES) ~ COHO2 + COHO3 + W, data = d, model = 1,
prate = list(COHO3 = 1, W = 1))
head(fit$rates) ID Time Events Censored RiskSet Surv.F CumRate BaselineRate
1 0 0 0 0 600 1.0000000 0.000000000 0.000000000
2 0 2 2 0 600 0.9950713 0.004940857 0.001038902
3 0 3 5 1 597 0.9827884 0.017361412 0.005223277
4 0 4 9 2 590 0.9607712 0.040018982 0.009528300
5 0 5 3 0 581 0.9534437 0.047674879 0.003219573
6 0 6 10 1 577 0.9292075 0.073423260 0.0108280946.18.2 Investigating Events
Event sequence data are given as a list of column groups, one per
sequence data structure – the manual’s seq8.cf defines two
from seq.d4, Y0..Y5 and S0..S5.
tda_seqev() counts the events (Box 3), and
tda_seqevd() their distribution over time for the first
structure (Box 4).
Box 176seq_info, seqev and seqevd
seqd4 <- read.table(ex("seq.d4"))
names(seqd4) <- c("ID", paste0("Y", 0:5), paste0("S", 0:5), "V1", "V2")
multi <- list(paste0("Y", 0:5), paste0("S", 0:5))
tda_seq_info(multi, data = seqd4) sn type variables tmin tmax nstates states
1 1 1 6 0 5 2 1, 2
2 2 1 6 0 5 2 1, 3tda_seqev(multi, data = seqd4) from to count
1 1 2 4
2 2 1 3tda_seqevd(multi, data = seqd4) time cases ev1_2 ev2_1 total
1 1 3 1 0 1
2 2 3 1 2 3
3 3 3 0 0 0
4 4 3 1 1 2
5 5 3 1 0 16.18.3 Data Generation
tda_seqmd() turns sequences into episode data around
chosen events, which is how a sequence becomes something a rate model
can take.
Box 177Episodes derived from sequences
tda_seqmd(paste0("Y", 0:5), event = c(1, 2), data = seqd4, summary = TRUE) time riskset events
1 1 2 1
2 2 1 1
3 3 2 0
4 4 2 1
5 5 2 1tda_seqmd(paste0("Y", 0:5), event = c(1, 2), data = seqd4) id time event period1 period2 period3 period4 period5
1 1 1 0 1 0 0 0 0
2 1 2 1 0 1 0 0 0
3 1 5 0 0 0 0 0 1
4 2 1 1 1 0 0 0 0
5 2 3 0 0 0 1 0 0
6 2 4 0 0 0 0 1 0
7 2 5 1 0 0 0 0 1
8 3 3 0 0 0 1 0 0
9 3 4 1 0 0 0 1 0tda_seqmd(paste0("Y", 0:5), event = c(1, 2), data = seqd4,
covariates = c("V1", "V2")) id time event period1 period2 period3 period4 period5 V1 V2
1 1 1 0 1 0 0 0 0 1 -10
2 1 2 1 0 1 0 0 0 1 -10
3 1 5 0 0 0 0 0 1 1 -10
4 2 1 1 1 0 0 0 0 2 -15
5 2 3 0 0 0 1 0 0 2 -15
6 2 4 0 0 0 0 1 0 2 -15
7 2 5 1 0 0 0 0 1 2 -15
8 3 3 0 0 0 1 0 0 2 -20
9 3 4 1 0 0 0 1 0 2 -20xe= adds time-varying covariates derived from a date:
xe=[V1],[V2] (the manual’s seq8c.cf, Box 4)
gives, for each episode, the process time since the date each variable
holds – event_covariates=, named after the columns TDA
writes.
Box 178seqmd with xe= (seq8c.cf)
tda_seqmd(paste0("Y", 0:5), event = c(1, 2), data = seqd4,
covariates = c("V1", "V2"),
event_covariates = list(V1_D = "V1", V2_D = "V2")) id time event period1 period2 period3 period4 period5 V1 V2 V1_D V2_D
1 1 1 0 1 0 0 0 0 1 -10 0 11
2 1 2 1 0 1 0 0 0 1 -10 1 12
3 1 5 0 0 0 0 0 1 1 -10 4 15
4 2 1 1 1 0 0 0 0 2 -15 -1 16
5 2 3 0 0 0 1 0 0 2 -15 1 18
6 2 4 0 0 0 0 1 0 2 -15 2 19
7 2 5 1 0 0 0 0 1 2 -15 3 20
8 3 3 0 0 0 1 0 0 2 -20 1 23
9 3 4 1 0 0 0 1 0 2 -20 2 24A third form of xe= counts the events of a given type in
another sequence structure before the present time and the time since
the last one, with effect-shape dummies (seq8d.cf, the
manual’s Box 5): [1,2,1,4] is the event 1 -> 2 in
structure 1, with 4 dummies.
Box 179seqmd with event counting (seq8d.cf)
tda_seqmd(paste0("Y", 0:5), event = c(1, 2), data = seqd4,
covariates = c("V1", "V2"),
event_covariates = list(V1_D = "V1", V2_D = "V2"),
options = list(xe = "[V1],[V2],[1,2,1,4]")) id time event period1 period2 period3 period4 period5 V1 V2 V1_D V2_D E_2_1 ED_2_1 E0_2_1 E1_2_1
1 1 1 0 1 0 0 0 0 1 -10 0 11 0 -1 0 0
2 1 2 1 0 1 0 0 0 1 -10 1 12 0 -1 0 0
3 1 5 0 0 0 0 0 1 1 -10 4 15 1 4 0 1
4 2 1 1 1 0 0 0 0 2 -15 -1 16 0 -1 0 0
5 2 3 0 0 0 1 0 0 2 -15 1 18 1 2 0 1
6 2 4 0 0 0 0 1 0 2 -15 2 19 1 2 0 0
7 2 5 1 0 0 0 0 1 2 -15 3 20 1 2 0 0
8 3 3 0 0 0 1 0 0 2 -20 1 23 1 2 0 1
9 3 4 1 0 0 0 1 0 2 -20 2 24 1 2 0 0
E2_2_1 E3_2_1 E4_2_1
1 0 0 0
2 0 0 0
3 0 0 0
4 0 0 0
5 0 0 0
6 1 0 0
7 0 1 0
8 0 0 0
9 1 0 06.19.1 Contingency Tables
Box 180loglin on a contingency table
ll1 <- read.table(ex("ll1.dat"), col.names = c("X1", "X2", "H"))
tda_loglin(~ X1 + X2, data = ll1, weights = "H")$table Idx X1 X2 H
1 1 1 2 3
2 2 1 3 0
3 3 1 4 0
4 4 2 2 0
5 5 2 3 0
6 6 2 4 3
7 7 5 2 0
8 8 5 3 2
9 9 5 4 06.19.2 Loglinear Models
ll2.dat carries a fourth column, S, the
scale factors of 6.19.1’s structural zeros; ll2.cf defines
it and does not use it, and the residual table’s Scale
column is all ones accordingly. scale= would pass it.
Box 181A loglinear model with residuals
ll2 <- read.table(ex("ll2.dat"), col.names = c("X1", "X2", "F", "S"))
fit <- tda_loglin(~ X1 + X2, data = ll2, weights = "F", residuals = TRUE)
fitCall: tda_loglin(formula = ~X1 + X2, data = ll2, weights = "F", residuals = TRUE)
Cases: 6
Idx X1 X2 F
1 1 1 3
2 1 2 4
3 2 1 0
4 2 2 2
5 3 1 3
6 3 2 5
Model: ~X1 + X2
Likelihood Ratio Statistic: 1.928719 Prob: 0.9643594
Degrees of Freedom: 2
idx parameter coeff error t_stat signif
0 Constant 0.8344470 NA NA NA
1 X1[1] 0.3730772 0.3646046 1.023238 0.6938046
2 X1[2] -0.8796858 0.5019802 -1.752431 0.9203003
3 X2[1] -0.3030679 0.2537596 -1.194311 0.7676437coef(fit) Constant X1[1] X1[2] X2[1]
0.8344470 0.3730772 -0.8796858 -0.3030679 head(residuals(fit)) Idx X1 X2 Observed Fitted Residual Scale
1 1 1 1 3 2.4705882 0.5459413 1
2 2 1 2 4 4.5294118 -0.5459413 1
3 3 2 1 0 0.7058824 -1.1119189 1
4 4 2 2 2 1.2941176 1.1119189 1
5 5 3 1 3 2.8235294 0.1794351 1
6 6 3 2 5 5.1764706 -0.1794351 17 Relational Data
The manual’s example graph is gd1.dat: an edge list
I, J with two value variables, V1 and
V2, so every node count comes once per value variable.
tda_graph() is the gdd command.
7.2.1.1 Degree of Nodes
gni reports each node of the current graph: its number,
its label, and per value variable the in-degree, out-degree and number
of loops; tda_g_degrees() is gni, and
gd4.cf runs it on gd1.dat. A loop (node 8 to
itself) is counted in its own column and not in the degrees: the
manual’s Box 1 prints node 8’s first-graph degrees as 1 and 1, while
gd4.cf run through TDA gives 0 and 0, as does a build of
Rohwer’s unmodified sources, so the box is out of date.
Box 182gni: in-degree, out-degree and loops per value variable (gd4.cf)
gd1 <- read.table(ex("gd1.dat"), col.names = c("I", "J", "V1", "V2"))
g1 <- tda_graph(gd1, directed = TRUE)
tda_g_degrees(g1) index node in_degree1 out_degree1 loops1 in_degree2 out_degree2 loops2 degree
1 1 1 1 2 0 2 0 0 3
2 2 5 1 0 0 0 1 0 1
3 3 7 1 1 0 1 1 0 2
4 4 8 0 0 1 0 1 0 0
5 5 9 0 0 0 0 0 0 0
6 6 11 0 1 0 1 0 0 1
7 7 12 1 0 0 0 1 0 17.2.1.2 Direct Links
tda_g_links() is gdln: for each node the
nodes it links to, one ragged record per node with any forward
links.
Box 183gdln: forward links
tda_g_links(g1)$table # forward links index node n_links links
1 1 1 2 5, 7
2 3 7 1 1
3 4 8 1 8
4 6 11 1 12tda_g_links(g1, backward = TRUE)$table # and backward V1 V2 V3 V4
1 1 1 1 7
2 2 5 1 1
3 3 7 1 1
4 4 8 1 8
5 7 12 1 117.2.2.1 Topological Sorting
tda_g_toposort() is gsort, on the manual’s
gd6.dat: a directed acyclic graph, its nodes in an order
that respects every edge.
Box 184gsort (gd22.cf)
gd6 <- read.table(ex("gd6.dat"), col.names = c("I", "J", "V"))
gsrt <- tda_graph(gd6, directed = TRUE)
tda_g_toposort(gsrt)$table i node_i label_i
1 1 1 4
2 2 2 5
3 3 4 1
4 4 5 2
5 5 7 3# option 2 relabels the graph's edges, keeping their values
tda_g_toposort(gsrt, relabel_graph = TRUE)$table label_i label_j node_i node_j value
1 1 2 4 5 6
2 1 3 4 7 5
3 1 5 4 2 7
4 2 3 5 7 1
5 2 4 5 1 3
6 3 4 7 1 2
7 3 5 7 2 4
8 4 5 1 2 87.2.3.1 Connected Components
tda_g_components() is gcon.
gd7.cf declares gd1.dat undirected and valued
(gt = 2) and asks for the components of its second graph,
the one V2 defines (gn = 2): node 8 joins
component 1 through the edge 8-7 that V2 carries and
V1 does not.
Box 185gcon: components of graph 2 (gd7.cf)
gcomp <- tda_graph(gd1, directed = FALSE)
tda_g_components(gcomp, gn = 2)$table component n i node_i
1 1 4 1 1
2 1 4 2 5
3 1 4 3 7
4 1 4 4 8
5 2 1 5 9
6 3 2 6 11
7 3 2 7 12# option 2 lists each component's edges instead, with their values
tda_g_components(gcomp, gn = 2, edges = TRUE)$table component n i j node_i node_j value
1 1 4 1 2 1 5 4
2 1 4 1 3 1 7 15
3 1 4 3 4 7 8 0
4 2 1 5 5 9 9 -1
5 3 2 6 7 11 12 147.2.3.2 Reachable Nodes in Directed Graphs
gdcon (tda_g(g, "gdcon")) is the directed
counterpart of gcon: strongly connected components
(opt = 1) or, with opt = 2, the nodes each
node reaches. gd8.cf runs it on graph 2 of
gd1.dat.
Box 186gdcon: reachable nodes (gd8.cf)
7.2.3.3 Cut Nodes and Blocks
tda_g_cutpoints() is gcut, on
gd8.dat: the nodes whose removal disconnects the graph, and
with opt = 2 the blocks.
Box 187gcut (gd26.cf)
gd8 <- read.table(ex("gd8.dat"), col.names = c("I", "J", "V"))
g8 <- tda_graph(gd8, directed = FALSE)
tda_g_cutpoints(g8)$table2 records, lengths 3-5
[1] 1 5 2 2 4
[2] 2 3 0 tda_g_cutpoints(g8, opt = 2)$table V1 V2 V3 V4 V5
1 1 1 1 2 0
2 1 2 2 2 1
3 1 3 3 2 0
4 1 4 4 2 1
5 1 5 5 2 0
6 2 6 6 0 0
7 2 7 7 0 0
8 2 8 8 0 07.2.4.1 Enumeration of Paths
tda_g_paths() is gep, on
gd5.dat, a small directed graph; gd20.cf runs
its six output options: the paths between all pairs of nodes, their
number per pair, the shortest and longest path per pair, and the two
full listings with each path’s length and value.
Box 188gep (gd20.cf)
gd5 <- read.table(ex("gd5.dat"), col.names = c("I", "J", "V"))
g5 <- tda_graph(gd5, directed = TRUE)
for (o in 1:6)
print(tda_g_paths(g5, opt = o)$table) V1 V2 V3 V4 V5 V6 V7 V8 V9 V10
1 1 2 1 2 1 1 1 1.0 1 1.0
2 1 3 1 3 3 1 2 3.0 4 13.0
3 1 4 1 4 3 1 2 4.0 4 13.5
4 1 5 1 5 3 1 2 1.5 4 14.0
5 2 1 2 1 3 1 2 8.0 4 18.0
6 2 3 2 3 3 1 1 2.0 3 12.0
7 2 4 2 4 3 1 1 3.0 3 12.5
8 2 5 2 5 3 1 1 0.5 3 13.0
9 3 1 3 1 1 1 1 6.0 1 6.0
10 3 2 3 2 2 1 1 0.5 2 7.0
11 3 4 3 4 3 1 2 3.5 3 10.0
12 3 5 3 5 5 1 2 1.0 4 14.0
13 4 1 4 1 1 1 3 15.0 3 15.0
14 4 2 4 2 2 1 3 9.5 4 16.0
15 4 3 4 3 1 1 2 9.0 2 9.0
16 4 5 4 5 1 1 1 4.0 1 4.0
17 5 1 5 1 1 1 2 11.0 2 11.0
18 5 2 5 2 2 1 2 5.5 3 12.0
19 5 3 5 3 1 1 1 5.0 1 5.0
20 5 4 5 4 3 1 3 8.5 4 15.0
45 records, lengths 9-12
[1] 1 2 1 2 1 1 1 1 2
[2] 1 3 1 3 1 2 3 1 2 3
[3] 1 3 1 3 2 4 13 1 2 4 5 3
[4] 1.0 3.0 1.0 3.0 3.0 3.0 6.5 1.0 2.0 5.0 3.0
[5] 1 4 1 4 1 3 10 1 2 3 4
[6] 1 4 1 4 2 2 4 1 2 4
[7] 1.0 4.0 1.0 4.0 3.0 4.0 13.5 1.0 2.0 5.0 3.0 4.0
[8] 1 5 1 5 1 4 14 1 2 3 4 5
[9] 1 5 1 5 2 3 8 1 2 4 5
[10] 1.0 5.0 1.0 5.0 3.0 2.0 1.5 1.0 2.0 5.0
... and 35 more
V1 V2 V3 V4 V5 V6 V7
1 1 1 0 1.0 3 4.0 1.5
2 2 2 8 0.0 2 3.0 0.5
3 3 3 6 0.5 0 3.5 1.0
4 4 4 15 9.5 9 0.0 4.0
5 5 5 11 5.5 5 8.5 0.0
V1 V2 V3 V4 V5 V6 V7
1 1 1 0 1 13 13.5 14
2 2 2 18 0 12 12.5 13
3 3 3 6 7 0 10.0 14
4 4 4 15 16 9 0.0 4
5 5 5 11 12 5 15.0 0
V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13
1 1 2 1 2 1 1 1 1.0 0 0 0 0 0
2 1 3 1 3 3 1 2 3.0 0 1 0 0 0
3 1 4 1 4 3 1 2 4.0 0 1 0 0 0
4 1 5 1 5 3 1 2 1.5 0 1 0 0 0
5 2 1 2 1 3 1 2 8.0 0 0 1 0 0
6 2 3 2 3 3 1 1 2.0 0 0 0 0 0
7 2 4 2 4 3 1 1 3.0 0 0 0 0 0
8 2 5 2 5 3 1 1 0.5 0 0 0 0 0
9 3 1 3 1 1 1 1 6.0 0 0 0 0 0
10 3 2 3 2 2 1 1 0.5 0 0 0 0 0
11 3 4 3 4 3 1 2 3.5 0 1 0 0 0
12 3 5 3 5 5 1 2 1.0 0 1 0 0 0
13 4 1 4 1 1 1 3 15.0 0 0 1 0 1
14 4 2 4 2 2 1 3 9.5 0 0 1 0 1
15 4 3 4 3 1 1 2 9.0 0 0 0 0 1
16 4 5 4 5 1 1 1 4.0 0 0 0 0 0
17 5 1 5 1 1 1 2 11.0 0 0 1 0 0
18 5 2 5 2 2 1 2 5.5 0 0 1 0 0
19 5 3 5 3 1 1 1 5.0 0 0 0 0 0
20 5 4 5 4 3 1 3 8.5 0 1 1 0 0
V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13
1 1 2 1 2 1 1 1 1.0 0 0 0 0 0
2 1 3 1 3 3 1 2 3.0 0 1 0 0 0
3 1 3 1 3 3 2 3 6.5 0 1 0 0 1
4 1 3 1 3 3 3 4 13.0 0 1 0 1 1
5 1 4 1 4 3 1 2 4.0 0 1 0 0 0
6 1 4 1 4 3 2 3 10.0 0 1 1 0 0
7 1 4 1 4 3 3 4 13.5 0 1 1 0 1
8 1 5 1 5 3 1 2 1.5 0 1 0 0 0
9 1 5 1 5 3 2 3 8.0 0 1 0 1 0
10 1 5 1 5 3 3 4 14.0 0 1 1 1 0
11 2 1 2 1 3 1 2 8.0 0 0 1 0 0
12 2 1 2 1 3 2 3 11.5 0 0 1 0 1
13 2 1 2 1 3 3 4 18.0 0 0 1 1 1
14 2 3 2 3 3 1 1 2.0 0 0 0 0 0
15 2 3 2 3 3 2 2 5.5 0 0 0 0 1
16 2 3 2 3 3 3 3 12.0 0 0 0 1 1
17 2 4 2 4 3 1 1 3.0 0 0 0 0 0
18 2 4 2 4 3 2 2 9.0 0 0 1 0 0
19 2 4 2 4 3 3 3 12.5 0 0 1 0 1
20 2 5 2 5 3 1 1 0.5 0 0 0 0 0
21 2 5 2 5 3 2 2 7.0 0 0 0 1 0
22 2 5 2 5 3 3 3 13.0 0 0 1 1 0
23 3 1 3 1 1 1 1 6.0 0 0 0 0 0
24 3 2 3 2 2 1 1 0.5 0 0 0 0 0
25 3 2 3 2 2 2 2 7.0 1 0 0 0 0
26 3 4 3 4 3 1 2 3.5 0 1 0 0 0
27 3 4 3 4 3 2 1 7.0 0 0 0 0 0
28 3 4 3 4 3 3 3 10.0 1 1 0 0 0
29 3 5 3 5 5 1 2 1.0 0 1 0 0 0
30 3 5 3 5 5 2 3 7.5 1 1 0 0 0
31 3 5 3 5 5 3 3 7.5 0 1 0 1 0
32 3 5 3 5 5 4 2 11.0 0 0 0 1 0
33 3 5 3 5 5 5 4 14.0 1 1 0 1 0
34 4 1 4 1 1 1 3 15.0 0 0 1 0 1
35 4 2 4 2 2 1 3 9.5 0 0 1 0 1
36 4 2 4 2 2 2 4 16.0 1 0 1 0 1
37 4 3 4 3 1 1 2 9.0 0 0 0 0 1
38 4 5 4 5 1 1 1 4.0 0 0 0 0 0
39 5 1 5 1 1 1 2 11.0 0 0 1 0 0
40 5 2 5 2 2 1 2 5.5 0 0 1 0 0
41 5 2 5 2 2 2 3 12.0 1 0 1 0 0
42 5 3 5 3 1 1 1 5.0 0 0 0 0 0
43 5 4 5 4 3 1 3 8.5 0 1 1 0 0
44 5 4 5 4 3 2 2 12.0 0 0 1 0 0
45 5 4 5 4 3 3 4 15.0 1 1 1 0 0The graph itself, as the manual draws it (gd18.cf): five
nodes, curved arrows for the edges that run both ways, the edge values
as labels.
Box 189The directed graph of gd5.dat, drawn (gd18.cf)
p <- tda_ps(xlim = c(0, 8), ylim = c(-1, 5), width = 80, height = 40)
nodes <- data.frame(id = 1:5, x = c(3, 5, 1, 7, 7), y = c(3, 1, 1, 4, 1),
grey = 0.8)
edges <- data.frame(from = c(1, 2, 3, 3, 2, 4, 5, 2, 3),
to = c(2, 3, 2, 1, 4, 5, 3, 5, 4),
curve = c(-3, 0, -3, -3, 0, 0, -3, 0, -3),
label = c(1, 2, 0.5, 6, 3, 4, 5, 0.5, 7),
arrow = "1.5,1.0")
plot(tda_pl_graph(p, nodes, edges))
7.2.4.2 All Shortest Paths
tda_g_shortest() is gsp, the same graph,
gd27.cf.
Box 190gsp (gd27.cf)
tda_g_shortest(g5, opt = 1, fmt = "4.1")$table index node to1 dist1 to2 dist2 to3 dist3 to4 dist4
1 1 1 2 1 3 3.0 4 4.0 5 1.5
2 2 2 1 8 3 2.0 4 3.0 5 0.5
3 3 3 1 6 2 0.5 4 3.5 5 1.0
4 4 4 1 15 2 9.5 3 9.0 5 4.0
5 5 5 1 11 2 5.5 3 5.0 4 8.57.2.4.3 Transitive Closure
tda_g_transitive() is gtcl;
gd16.cf runs it on gd5.dat read as an
undirected, an undirected valued and a directed graph,
opt = 2 for the closure as a matrix.
Box 191gtcl (gd16.cf)
# gdd's gt= is the graph type: 1 undirected unvalued, 2 undirected
# valued, 3 directed unvalued, 4 directed valued. The manual's box runs
# gtcl over all four.
for (gt in 1:4)
print(tda_g_transitive(tda_graph(gd5, options = list(gt = gt)),
opt = 2, fmt = "4.1")$table) index node n1 n2 n3 n4 n5
1 1 1 0 1 1 1 1
2 2 2 1 0 1 1 1
3 3 3 1 1 0 1 1
4 4 4 1 1 1 0 1
5 5 5 1 1 1 1 0
index node n1 n2 n3 n4 n5
1 1 1 0.0 1.0 1.5 4.0 1.5
2 2 2 1.0 0.0 0.5 3.0 0.5
3 3 3 1.5 0.5 0.0 3.5 1.0
4 4 4 4.0 3.0 3.5 0.0 3.5
5 5 5 1.5 0.5 1.0 3.5 0.0
index node n1 n2 n3 n4 n5
1 1 1 0 1 1 1 1
2 2 2 1 0 1 1 1
3 3 3 1 1 0 1 1
4 4 4 1 1 1 0 1
5 5 5 1 1 1 1 0
index node n1 n2 n3 n4 n5
1 1 1 0 1.0 3 4.0 1.5
2 2 2 8 0.0 2 3.0 0.5
3 3 3 6 0.5 0 3.5 1.0
4 4 4 15 9.5 9 0.0 4.0
5 5 5 11 5.5 5 8.5 0.07.2.5.1 Depth-first Spanning Trees
tda_g_spanning() is gst, on
gd7.dat: one spanning tree per component, the edges listed
with their values.
Box 192gst (gd24.cf)
gd7 <- read.table(ex("gd7.dat"), col.names = c("I", "J", "V"))
tda_g_spanning(tda_graph(gd7, directed = FALSE))$table tree node_i node_j value
1 1 1 2 1
2 1 2 3 7
3 1 3 4 5
4 1 4 5 3
5 2 6 7 9
6 2 7 8 10The manual’s Figure 7.2.5.1-1 is gd7.dat’s first
component drawn by gd21.cf, with its spanning tree’s
edges.
Box 193gd7.dat drawn (gd21.cf)
p <- tda_ps(xlim = c(0, 6), ylim = c(0, 5), width = 60, height = 30)
nodes <- data.frame(id = c(5, 7, 4, 1, 2), x = c(1, 3, 5, 2, 4),
y = c(3, 3, 3, 1, 1), grey = 0.8)
edges <- data.frame(from = c(5, 4, 5, 7, 7, 4, 1, 4),
to = c(7, 7, 1, 1, 2, 2, 2, 5),
curve = c(0, 0, 0, 0, 0, 0, 0, 3),
label = c(1, 5, 3, 2, 4, 7, 8, 6),
arrow = "1.5,1.0")
plot(tda_pl_graph(p, nodes, edges))
Box 194gd23.cf
p <- tda_ps(xlim = c(0, 7), ylim = c(0, 3.7), width = 60, height = 30)
nodes <- data.frame(id = 1:8, x = c(0.5, 2, 2, 3.5, 4.5, 5, 6, 6),
y = c(1.5, 2.5, 0.5, 1.5, 0.5, 1.5, 0.5, 2.5), grey = 0.8)
edges <- data.frame(
from = c(1, 2, 4, 5, 3, 1, 2, 6, 6, 8),
to = c(2, 4, 5, 3, 4, 3, 3, 8, 7, 7),
label = 1:10,
arrow = "1.5,1.0")
p <- tda_pl_graph(p, nodes, edges)
plot(p)
7.2.5.2 Minimum Spanning Trees
tda_g_mst() is gmst, gd25.cf,
on the same gd7.dat.
Box 195gmst (gd25.cf)
7.2.5.3 Enumeration of Spanning Trees
tda_g_spantrees() is gnst, on
gd10.dat (gd32.cf): the number of spanning
trees, and with opt = 2 the trees themselves.
Box 196gnst (gd32.cf)
gd10 <- read.table(ex("gd10.dat"), col.names = c("I", "J", "V"))
g10 <- tda_graph(gd10, directed = FALSE)
tda_g_spantrees(g10, opt = 1)$table V1 V2 V3 V4 V5 V6
1 1 1 0 1 1 1
2 1 2 0 1 1 3
3 1 3 0 1 2 1
4 1 4 0 1 2 3
5 1 5 0 1 4 1
6 1 6 0 3 1 1
7 1 7 0 3 1 3
8 1 8 0 3 4 1tda_g_spantrees(g10, opt = 2)$table V1 V2 V3 V4 V5 V6
1 1 1 3 1 2 1
2 1 1 3 2 3 1
3 1 1 3 3 4 1
4 1 2 3 1 2 1
5 1 2 3 2 3 1
6 1 2 3 3 4 3
7 1 3 3 1 2 1
8 1 3 3 2 3 2
9 1 3 3 3 4 1
10 1 4 3 1 2 1
11 1 4 3 2 3 2
12 1 4 3 3 4 3
13 1 5 3 1 2 1
14 1 5 3 2 3 4
15 1 5 3 3 4 1
16 1 6 3 1 2 3
17 1 6 3 2 3 1
18 1 6 3 3 4 1
19 1 7 3 1 2 3
20 1 7 3 2 3 1
21 1 7 3 3 4 3
22 1 8 3 1 2 3
23 1 8 3 2 3 4
24 1 8 3 3 4 1Box 197gd31.cf
p <- tda_ps(xlim = c(0, 8), ylim = c(-1, 5), width = 80, height = 40)
nodes <- data.frame(id = 1:4, x = c(3, 5, 3, 5), y = c(3, 3, 1, 1), grey = 0.8)
edges <- data.frame(from = c(1, 1, 1, 2, 3), to = c(2, 3, 4, 3, 4))
p <- tda_pl_graph(p, nodes, edges)
plot(p)
7.2.6.1 Fundamental Set of Cycles
tda_g_cycles() is gcyc;
gd30.cf runs it on gd9.dat, an undirected
graph with 6 nodes and 11 edges. opt = 1 lists a
fundamental set of cycles, one ragged record each: index, cycle number,
length, and the nodes.
gcyc’s four options are four views of the same cycle
space. Option 1 lists the nodes of each fundamental cycle, so its rows
are ragged; option 2 gives the edge and one indicator per fundamental
cycle; option 3 the same over every cycle found; option 4 writes the
edges first and then one indicator row per cycle, two row shapes in the
one file, which is why it arrives ragged as well.
Box 198gcyc, options 1 to 4
gd9 <- read.table(ex("gd9.dat"), col.names = c("I", "J", "V"))
g9 <- tda_graph(gd9, directed = FALSE)
tda_g_cycles(g9, "fundamental_nodes")$table6 records, lengths 5-6
[1] 1 1 6 1 3
[2] 1 2 5 6 1 2
[3] 1 3 5 6 4
[4] 1 4 4 6 1 2
[5] 1 5 4 6 1 3
[6] 1 6 3 1 2 tda_g_cycles(g9, "fundamental_edges")$table cycle index from to in1 in2 in3 in4 in5 in6
1 1 1 1 2 0 1 0 1 0 1
2 1 2 1 3 1 0 0 0 1 1
3 1 3 1 6 1 1 0 1 1 0
4 1 4 2 3 0 0 0 0 0 1
5 1 5 2 4 0 0 0 1 0 0
6 1 6 2 5 0 1 0 0 0 0
7 1 7 3 4 0 0 0 0 1 0
8 1 8 3 6 1 0 0 0 0 0
9 1 9 4 5 0 0 1 0 0 0
10 1 10 4 6 0 0 1 1 1 0
11 1 11 5 6 0 1 1 0 0 0tda_g_cycles(g9, "all_v1")$table cycle index from to in1 in2 in3 in4 in5 in6 in7 in8 in9 in10 in11 in12 in13 in14 in15 in16 in17
1 1 1 1 2 0 1 1 1 1 0 1 0 1 0 1 1 0 1 0 1 1
2 1 2 1 3 1 1 0 1 0 0 1 0 1 0 0 0 0 1 0 0 1
3 1 3 1 6 1 0 1 0 1 0 0 0 0 0 1 1 0 0 0 1 0
4 1 4 2 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
5 1 5 2 4 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0
6 1 6 2 5 0 1 1 1 1 0 0 1 0 1 0 0 0 1 0 1 1
7 1 7 3 4 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1
8 1 8 3 6 1 1 0 1 0 0 1 0 1 0 0 0 1 0 1 1 0
9 1 9 4 5 0 0 0 1 1 1 0 0 1 1 1 0 0 0 1 1 1
10 1 10 4 6 0 0 0 1 1 1 1 1 0 0 0 1 1 1 0 0 0
11 1 11 5 6 0 1 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0
in18 in19 in20 in21 in22 in23 in24 in25 in26 in27 in28 in29 in30 in31 in32 in33 in34 in35 in36
1 0 1 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 1 0
2 1 0 0 1 1 1 0 0 1 0 1 0 0 1 1 0 0 0 0
3 1 1 0 1 0 1 1 0 1 0 1 0 0 1 1 0 0 1 0
4 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1
5 0 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 1
6 0 0 1 1 0 0 0 1 1 1 1 0 0 0 0 1 1 0 0
7 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1
8 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 0 0 0 0
9 1 0 0 0 0 0 0 0 0 1 1 0 1 1 0 0 1 1 0
10 0 0 0 0 0 1 0 0 0 1 1 1 0 0 1 1 0 0 0
11 1 0 1 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 0
in37 in38
1 1 1
2 0 1
3 1 0
4 1 1
5 0 0
6 0 0
7 1 0
8 0 0
9 0 0
10 1 0
11 0 0# option 4's records are of two shapes -- a cycle's row and its edge
# list -- so they stay a ragged list; the manual prints all of them
print(tda_g_cycles(g9, "all_v2")$table, n = Inf)49 records, lengths 4-13
[1] 1 1 1 2
[2] 1 2 1 3
[3] 1 3 1 6
[4] 1 4 2 3
[5] 1 5 2 4
[6] 1 6 2 5
[7] 1 7 3 4
[8] 1 8 3 6
[9] 1 9 4 5
[10] 1 10 4 6
[11] 1 11 5 6
[12] 1 1 0 1 1 0 0 0 0 1 0 0 0
[13] 1 2 1 1 0 0 0 1 0 1 0 0 1
[14] 1 3 1 0 1 0 0 1 0 0 0 0 1
[15] 1 4 1 1 0 0 0 1 0 1 1 1 0
[16] 1 5 1 0 1 0 0 1 0 0 1 1 0
[17] 1 6 0 0 0 0 0 0 0 0 1 1 1
[18] 1 7 1 1 0 0 1 0 0 1 0 1 0
[19] 1 8 0 0 0 0 1 1 0 0 0 1 1
[20] 1 9 1 1 0 0 1 0 0 1 1 0 1
[21] 1 10 0 0 0 0 1 1 0 0 1 0 0
[22] 1 11 1 0 1 0 1 0 0 0 1 0 1
[23] 1 12 1 0 1 0 1 0 0 0 0 1 0
[24] 1 13 0 0 0 0 0 0 1 1 0 1 0
[25] 1 14 1 1 0 0 0 1 1 0 0 1 1
[26] 1 15 0 0 0 0 0 0 1 1 1 0 1
[27] 1 16 1 0 1 0 0 1 1 1 1 0 0
[28] 1 17 1 1 0 0 0 1 1 0 1 0 0
[29] 1 18 0 1 1 0 0 0 1 0 1 0 1
[30] 1 19 1 0 1 0 1 0 1 1 0 0 0
[31] 1 20 0 0 0 0 1 1 1 1 0 0 1
[32] 1 21 0 1 1 0 1 1 1 0 0 0 1
[33] 1 22 1 1 0 0 1 0 1 0 0 0 0
[34] 1 23 0 1 1 0 0 0 1 0 0 1 0
[35] 1 24 1 0 1 1 0 0 0 1 0 0 0
[36] 1 25 0 0 0 1 0 1 0 1 0 0 1
[37] 1 26 0 1 1 1 0 1 0 0 0 0 1
[38] 1 27 0 0 0 1 0 1 0 1 1 1 0
[39] 1 28 0 1 1 1 0 1 0 0 1 1 0
[40] 1 29 0 0 0 1 1 0 0 1 0 1 0
[41] 1 30 0 0 0 1 1 0 0 1 1 0 1
[42] 1 31 0 1 1 1 1 0 0 0 1 0 1
[43] 1 32 0 1 1 1 1 0 0 0 0 1 0
[44] 1 33 0 0 0 1 0 1 1 0 0 1 1
[45] 1 34 0 0 0 1 0 1 1 0 1 0 0
[46] 1 35 1 0 1 1 0 0 1 0 1 0 1
[47] 1 36 0 0 0 1 1 0 1 0 0 0 0
[48] 1 37 1 0 1 1 0 0 1 0 0 1 0
[49] 1 38 1 1 0 1 0 0 0 0 0 0 0 Box 199gd29.cf
p <- tda_ps(xlim = c(0, 8), ylim = c(-1, 5), width = 80, height = 40)
nodes <- data.frame(id = c(1, 3, 4, 6, 2, 5), x = c(3, 3, 5, 1, 5, 7),
y = c(3, 1, 1, 1, 3, 1), grey = 0.8)
edges <- data.frame(
from = c(1, 3, 1, 1, 3, 3, 2, 2, 4, 6, 6),
to = c(6, 6, 3, 2, 4, 2, 4, 5, 5, 4, 5),
curve = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 5))
p <- tda_pl_graph(p, nodes, edges)
plot(p)
7.2.6.2 Enumeration of Cycles in Directed Graphs
tda_g_dcycles() is gdcyc, on
gd5.dat (gd33.cf).
Box 200gdcyc (gd33.cf)
tda_g_dcycles(g5, opt = 1)$table7 records, lengths 4-7
[1] 1 3 1 2 3
[2] 2 5 1 2 4 5 3
[3] 3 4 1 2 5 3
[4] 4 2 2 3
[5] 5 4 2 4 5 3
[6] 6 3 2 5 3
[7] 7 3 3 4 5 tda_g_dcycles(g5, opt = 2)$table V1 V2 V3
1 1 3 1
2 1 3 2
3 1 3 3
4 2 5 1
5 2 5 2
6 2 5 4
7 2 5 5
8 2 5 3
9 3 4 1
10 3 4 2
11 3 4 5
12 3 4 3
13 4 2 2
14 4 2 3
15 5 4 2
16 5 4 4
17 5 4 5
18 5 4 3
19 6 3 2
20 6 3 5
21 6 3 3
22 7 3 3
23 7 3 4
24 7 3 57.2.8.1 Maximal Flows
tda_g_flow() is gflow, on
gd11.dat: for every ordered pair of nodes the maximal flow,
one record per pair – index, the two nodes’ indices, the two nodes, the
flow.
Box 201gflow (gd35.cf)
gd11 <- read.table(ex("gd11.dat"), col.names = c("I", "J", "V"))
g11 <- tda_graph(gd11, directed = TRUE)
tda_g_flow(g11)$table index i j node_i node_j flow
1 1 1 2 1 2 30
2 2 1 3 1 3 27
3 3 1 4 1 4 17
4 4 1 5 1 5 17
5 5 1 6 1 6 30
6 6 1 7 1 7 30
7 7 2 3 2 3 27
8 8 2 4 2 4 17
9 9 2 5 2 5 17
10 10 2 6 2 6 37
11 11 2 7 2 7 47
12 12 3 7 3 7 22
13 13 4 3 4 3 50
14 14 4 5 4 5 54
15 15 4 7 4 7 22
16 16 5 3 5 3 50
17 17 5 7 5 7 22
18 18 6 3 6 3 17
19 19 6 4 6 4 17
20 20 6 5 6 5 17
21 21 6 7 6 7 49
22 22 8 1 8 1 40
23 23 8 2 8 2 30
24 24 8 3 8 3 47
25 25 8 4 8 4 37
26 26 8 5 8 5 37
27 27 8 6 8 6 30
28 28 8 7 8 7 50# option 2 gives the same flows as a matrix
tda_g_flow(g11, matrix = TRUE)$table V1 V2 V3 V4 V5 V6 V7 V8 V9 V10
1 1 1 0 30 27 17 17 30 30 -1
2 2 2 -1 0 27 17 17 37 47 -1
3 3 3 -1 -1 0 -1 -1 -1 22 -1
4 4 4 -1 -1 50 0 54 -1 22 -1
5 5 5 -1 -1 50 -1 0 -1 22 -1
6 6 6 -1 -1 17 17 17 0 49 -1
7 7 7 -1 -1 -1 -1 -1 -1 0 -1
8 8 8 40 30 47 37 37 30 50 0Box 202gd34.cf
p <- tda_ps(xlim = c(0, 11), ylim = c(-1, 7), width = 80, height = 40)
nodes <- data.frame(id = c(4, 1, 8, 6, 3, 7, 2, 5),
x = c(3, 3, 1, 8, 8, 10, 6, 4.5),
y = c(1, 5, 3, 1, 5, 3, 5, 3), grey = 0.8)
edges <- data.frame(
from = c(8, 8, 1, 6, 2, 6, 3, 2, 4, 5),
to = c(1, 4, 2, 4, 3, 7, 7, 6, 5, 3),
label = c(40, 20, 30, 17, 10, 32, 22, 37, 54, 50),
arrow = "1.5,1.0")
p <- tda_pl_graph(p, nodes, edges)
plot(p)
7.2.9.1 Cliques
tda_g_cliques() is gcliq, on
gd16.dat; the manual’s Box 1 runs both of its algorithms,
alg = 1 (Bron-Kerbosch) and alg = 2
(Harary-Ross), whose listings differ in order, not in content.
Box 203gcliq (gd42.cf)
gd16 <- read.table(ex("gd16.dat"), col.names = c("I", "J", "V"))
g16 <- tda_graph(gd16, directed = FALSE)
tda_g_cliques(g16, algorithm = "bron_kerbosch")$table4 records, lengths 6-7
[1] 1 1 4 3 2 1 4
[2] 1 2 3 3 2 5
[3] 1 3 3 3 6 4
[4] 1 4 3 3 6 5 tda_g_cliques(g16, algorithm = "harary_ross")$table5 records, lengths 6-7
[1] 1 1 4 1 2 3 4
[2] 1 2 3 4 2 3
[3] 1 3 3 3 2 5
[4] 1 4 3 4 3 6
[5] 1 5 3 3 5 6 Box 204gd43.cf
p <- tda_ps(xlim = c(0, 8), ylim = c(-1.5, 7), width = 80, height = 40)
nodes <- data.frame(id = 1:6, x = c(1, 3, 5, 3, 7, 7), y = c(3, 6, 3, 0, 6, 0),
grey = 0.8)
edges <- data.frame(
from = c(1, 1, 1, 2, 2, 2, 4, 4, 3, 3, 5),
to = c(3, 2, 4, 4, 3, 5, 3, 6, 5, 6, 6))
p <- tda_pl_graph(p, nodes, edges)
plot(p)
7.2.9.2 Compact Sets
tda_g_independent() is giset, independent
sets on gd12.dat (gd37.cf), and
tda_g_compact() is gcset, compact sets on
gd13.dat (gd39.cf).
Box 205giset and gcset
gd12 <- read.table(ex("gd12.dat"), col.names = c("I", "J", "V"))
tda_g_independent(tda_graph(gd12, directed = FALSE))$table V1 V2 V3
1 1 3 3
2 2 6 6gd13 <- read.table(ex("gd13.dat"), col.names = c("I", "J", "V"))
g13 <- tda_graph(gd13, directed = FALSE)
print(tda_g_compact(g13, opt = 1)$table, n = Inf) # the sets themselves3 records, lengths 7-9
[1] 1 1 2 1 3 1 3
[2] 1 2 2 2 8 4 6
[3] 1 3 4 7 8 1 2 3 5 tda_g_compact(g13, opt = 2)$table # one row per node V1 V2 V3 V4 V5 V6
1 1 1 2 1 3 1
2 1 1 2 1 3 3
3 1 2 2 2 8 4
4 1 2 2 2 8 6
5 1 3 4 7 8 1
6 1 3 4 7 8 2
7 1 3 4 7 8 3
8 1 3 4 7 8 5Box 206gd38.cf
p <- tda_ps(xlim = c(0, 8), ylim = c(-1.5, 7), width = 80, height = 40)
nodes <- data.frame(id = 1:6, x = c(1, 3, 5, 3, 7, 7), y = c(3, 6, 3, 0, 6, 0),
grey = 0.8)
edges <- data.frame(
from = c(1, 1, 1, 2, 2, 4, 4, 3, 3, 5),
to = c(3, 2, 4, 3, 5, 3, 6, 5, 6, 6),
label = c(1, 3, 12, 7, 5, 11, 2, 4, 15, 8))
p <- tda_pl_graph(p, nodes, edges)
plot(p)
7.2.9.3 Regular Equivalence
Box 207gd36.cf
p <- tda_ps(xlim = c(0, 8), ylim = c(-1.5, 7), width = 80, height = 40)
nodes <- data.frame(id = 1:6, x = c(1, 2.5, 4, 4, 5.5, 7),
y = c(1, 3, 1, 5, 3, 1), grey = 0.8)
edges <- data.frame(
from = c(1, 1, 2, 2, 2, 3, 3, 4, 5, 1),
to = c(2, 3, 3, 4, 5, 4, 5, 5, 6, 6),
curve = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 3))
p <- tda_pl_graph(p, nodes, edges)
plot(p)
7.3.1.1 The Linear Assignment Problem
tda_gap() is gap: given a cost matrix, the
assignment of rows to columns with minimal total cost. The manual’s
co1.dat is a 6 x 6 cost matrix, row by row; the result is
the permutation of its Box 2 and the matrix with its rows permuted
accordingly.
Box 208gap (co1.cf)
cost <- matrix(scan(ex("co1.dat"), quiet = TRUE), 6, 6, byrow = TRUE)
ap <- tda_gap(cost)
ap$assignment [,1] [,2]
[1,] 1 6
[2,] 2 4
[3,] 3 3
[4,] 4 1
[5,] 5 5
[6,] 6 2ap$permuted [,1] [,2] [,3] [,4] [,5] [,6]
[1,] 0 76 0 60 0 0
[2,] 24 0 18 0 60 40
[3,] 40 4 2 60 0 16
[4,] 75 3 18 0 55 27
[5,] 53 16 62 0 11 40
[6,] 16 84 10 28 0 47.3.1.2 The Quadratic Assignment Problem
tda_gqap() is gqap, on two symmetric
matrices, a flow and a distance matrix. co2.dat gives their
lower triangles in one file, one pair per line in the order (2,1),
(3,1), (3,2); the manual’s Box 2 is the permutation 2, 1, 3.
Box 209gqap (co2.cf)
co2 <- read.table(ex("co2.dat"), col.names = c("F", "D"))
lower <- function(v) { m <- matrix(0, 3, 3); m[lower.tri(m)] <- v; m + t(m) }
qap <- tda_gqap(lower(co2$F), lower(co2$D))
qap$assignment[1] 2 1 3qap$cost[1] 36# gqap's second output file: the multigraph under that permutation, with
# the flow and distance between each pair
qap$permuted i j node_i node_j flow distance
1 1 1 1 1 0 0
2 1 2 1 2 1 5
3 1 3 1 3 2 3
4 2 1 2 1 1 5
5 2 2 2 2 0 0
6 2 3 2 3 7 1
7 3 1 3 1 2 3
8 3 2 3 2 7 1
9 3 3 3 3 0 07.5.1.1 SAHN Algorithms
tda_cluster() covers TDA’s clustering commands. The
manual’s example dissimilarity matrix is cl1.dat, the upper
triangle of an 11 x 11 matrix, row by row; cl1.cf runs
hcls with opt = 2, complete linkage. The
output is the manual’s Box 3: at each level the two clusters merged,
named by their representative nodes.
Box 210hcls: complete-link clustering (cl1.cf)
D <- scan(ex("cl1.dat"), quiet = TRUE)
dm <- matrix(0, 11, 11)
dm[lower.tri(dm)] <- D
dm <- dm + t(dm)
tda_cluster(dm, method = "hierarchical_single", options = list(opt = 2))$table V1 V2 V3 V4
1 1 1 3 6
2 2 2 1 8
3 3 4 2 9
4 4 7 5 11
5 5 8 1 2
6 6 13 3 4
7 7 24 3 10
8 8 33 1 5
9 9 34 3 7
10 10 55 1 3cl1a.cf reads the dendrogram back and cuts it into four
levels with hclsp; tda_cutree() is both
steps.
Box 211hclsp: the dendrogram cut at four levels (cl1a.cf)
tda_cutree(dm, nlev = 4)$table node leaves parent
1 21 11 NA
2 19 6 21
3 15 2 21
4 5 1 21
5 11 1 21
6 16 4 21
7 13 2 21
8 14 2 21
9 20 5 21
10 7 1 21
11 18 4 21
12 10 1 21
13 17 3 217.5.1.2 Nearest-Neighbor Clustering
cl2.cf runs nncl twice: with a threshold
(sc = 10) and with a fixed number of neighbours
(ns = 2).
Box 212nncl (cl2.cf)
tda_cluster(dm, method = "nearest_neighbour", options = list(alg = 1, sc = 10))$table V1 V2 V3
1 1 1 1
2 2 2 1
3 3 3 2
4 4 4 1
5 5 5 3
6 6 6 2
7 7 7 4
8 8 8 1
9 9 9 1
10 10 10 5
11 11 11 3tda_cluster(dm, method = "nearest_neighbour", options = list(alg = 2, ns = 2))$table V1 V2 V3 V4 V5 V6
1 1 2 1 8 1 8
2 2 2 3 6 3 6
3 3 2 5 11 5 11
4 4 3 2 9 2 9
5 5 3 8 9 8 9
6 6 4 3 4 3 47.5.2.1 Maximally Different Cluster Centers
cl3.cf is hcld with its first algorithm:
divisive clustering around maximally different centres. The manual’s Box
2 lists each cluster’s nodes.
Box 213hcld (cl3.cf)
hc <- tda_cluster(dm, method = "hierarchical", algorithm = "centers")
hc$clusters$`1`
[1] 1 2 3 4 5 6 7 8 9 10 11
$`2`
[1] 3 4 6 7 10
$`3`
[1] 1 2 5 8 9 11
$`4`
[1] 4 7
$`5`
[1] 3 6 10
$`6`
[1] 1 2 8 9
$`7`
[1] 5 11
$`8`
[1] 3 6
$`9`
[1] 10
$`10`
[1] 1 8
$`11`
[1] 2 9hc$merges # one row per split: the two clusters, their sizes, their diameters V1 V2 V3 V4 V5
1 2 1 5 11 27.5
2 3 1 6 11 27.5
3 4 2 2 5 17.0
4 5 2 3 5 17.0
5 6 3 4 6 16.5
6 7 3 2 6 16.5
7 8 5 2 3 12.0
8 9 5 1 3 12.0
9 10 6 2 4 4.0
10 11 6 2 4 4.07.5.2.2 Maximally Homogeneous Clusters
The second algorithm splits on the diameter instead – at each step the cluster with the largest diameter is divided.
Box 214hcld, second algorithm
# the default stops after two splits; the manual's run goes on until
# the size bound is reached
hd <- tda_cluster(dm, method = "hierarchical", algorithm = "diameter",
max_splits = 5)
hd$clusters$`2`
[1] 33
$`3`
[1] 34
$`4`
[1] 24 3 3
$`5`
[1] 16 2 4
$`6`
[1] 8 4
$`7`
[1] 7 2
$`8`
[1] 1 2 3 6
$`9`
[1] 0 1 10
$`10`
[1] 0 1 4
$`11`
[1] 0 1 7hd$merges ci cj ni nj diameter_i diameter_j
1 2 1 6 11 33 55
2 3 1 5 11 34 55
3 4 3 3 5 24 34
4 5 3 2 5 16 34
5 6 2 4 6 8 33
6 7 2 2 6 7 33
7 8 4 2 3 1 24
8 9 4 1 3 0 24
9 10 5 1 2 0 16
10 11 5 1 2 0 16cl3p.cf draws the merge table as a dendrogram: columns 1
and 2 of cl3.df are the pair joined, column 5 the
distance.
Box 215cl3p.cf
7.5.3.1 The Bond Energy Algorithm
becl is the bond energy algorithm on a graph;
cl5.cf runs it on gd15.dat, five nodes, with
the four output options. Option 1 is the node permutation, option 2 the
edge list permuted (Box 3).
Box 216becl (cl5.cf)
gd15 <- read.table(ex("gd15.dat"), col.names = c("I", "J", "V"))
g15 <- tda_graph(gd15, directed = FALSE)
tda_g(g15, "becl", list(opt = 1, fmt = "3.0"))$table index node position node_at
1 1 3 1 3
2 2 2 2 2
3 3 5 3 5
4 4 4 4 4
5 5 1 5 1tda_g(g15, "becl", list(opt = 2, fmt = "3.0"))$table V1 V2 V3 V4 V5 V6 V7
1 1 2 3 2 3 2 1
2 3 4 5 4 5 4 1
3 3 5 5 1 5 1 1
4 4 5 4 1 4 1 1# options 3 and 4 take sc=, the value to fill a non-edge with; the
# default is -1 and cl5.cf asks for 0
tda_g(g15, "becl", list(opt = 3, fmt = "3.0", sc = 0))$table V1 V2 V3 V4 V5 V6 V7
1 2 1 2 3 2 3 1
2 3 1 5 3 5 3 0
3 3 2 5 2 5 2 0
4 4 1 4 3 4 3 0
5 4 2 4 2 4 2 0
6 4 3 4 5 4 5 1
7 5 1 1 3 1 3 0
8 5 2 1 2 1 2 0
9 5 3 1 5 1 5 1
10 5 4 1 4 1 4 1tda_g(g15, "becl", list(opt = 4, fmt = "3.0", sc = 0))$table V1 V2 V3 V4 V5
1 0 1 0 0 0
2 1 0 0 0 0
3 0 0 0 1 1
4 0 0 1 0 1
5 0 0 1 1 07.6.1.1 Direct and Indirect Control
tda_gfcf() is gfcf, control in a directed
valued graph – gd3.dat is a small ownership network. With
sc = 49 (control at more than 49 per cent) node 1 controls
nodes 2 to 9, node 9 controls 1 to 8, node 3 controls 5; the pair table
of the manual’s Box 2 is the result’s "pairs"
attribute.
Box 217gfcf (gd13.cf)
gd3 <- read.table(ex("gd3.dat"), col.names = c("I", "J", "V"))
ctl <- tda_gfcf(gd3, threshold = 49)
ctl node n_controlled controlled
1 1 8 2,3,4,5,6,7,8,9
2 2 0
3 3 1 5
4 4 0
5 5 0
6 6 0
7 7 0
8 8 0
9 9 8 1,2,3,4,5,6,7,8
10 10 0
11 11 0 head(attr(ctl, "pairs"), 8) i node controlled L S M R
1 1 1 2 1 50 8 8
2 1 1 3 1 50 8 8
3 1 1 4 4 60 8 8
4 1 1 5 2 50 8 8
5 1 1 6 3 60 8 8
6 1 1 7 2 60 8 8
7 1 1 8 3 60 8 8
8 1 1 9 5 50 8 8gbcf is the same question backwards – who controls a
given node – on gd3a.dat. It writes two files: the pair
table, and a summary with one row per node (the manual’s Box 4 shows
both).
Box 218gbcf (gd14.cf)
gd3a <- read.table(ex("gd3a.dat"), col.names = c("I", "J", "V"))
bwd <- tda_g_backward(tda_graph(gd3a, directed = TRUE), sc = 49)
bwd$table i node_i n_a node_j a_ji n_y node_k y_ki
1 2 2 1 1 50 1 1 50
2 3 3 1 1 50 1 1 50
3 4 4 4 2 30 2 1 60
4 4 4 4 3 10 2 10 55
5 4 4 4 8 20 2 -1 0
6 4 4 4 10 55 2 -1 0
7 5 5 1 3 50 1 1 50
8 6 6 3 3 30 2 1 60
9 6 6 3 7 30 2 11 22
10 6 6 3 11 22 2 -1 0
11 7 7 2 1 20 1 1 60
12 7 7 2 3 40 1 -1 0
13 8 8 2 5 30 1 1 60
14 8 8 2 7 30 1 -1 0
15 9 9 2 1 10 1 1 50
16 9 9 2 4 40 1 -1 0bwd$summary i node_i m
1 1 1 8
2 10 10 1
3 11 11 0# gd14a.cf asks for the second layout, opt = 2
tda_g_backward(tda_graph(gd3a, directed = TRUE), sc = 49,
opt = 2, fmt = "3.0")$table V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13
1 2 2 4 1 1 50 0 0 0 50 0 0 0
2 3 3 4 1 1 50 0 0 0 50 0 0 0
3 4 4 4 4 2 55 30 20 10 60 55 0 0
4 5 5 4 1 1 50 0 0 0 50 0 0 0
5 6 6 4 3 2 30 30 22 0 60 22 0 0
6 7 7 4 2 1 40 20 0 0 60 0 0 0
7 8 8 4 2 1 30 30 0 0 60 0 0 0
8 9 9 4 2 1 40 10 0 0 50 0 0 0In gd12.cf node 1 is drawn as a square,
shape = for plg’s gt=2, and
several edges are curved or run back to an earlier node.
Box 219gd12.cf
p <- tda_ps(xlim = c(0, 11), ylim = c(2.0, 7.5), width = 100, height = 50)
nodes <- data.frame(
id = c(9, 1, 3, 2, 4, 10, 7, 6, 5, 8, 11),
x = c(3, 3, 5, 5, 7, 9, 3, 5, 7, 9, 7),
y = c(7, 5, 5, 6, 5, 5, 3, 3, 4, 3, 3),
grey = 0.8,
shape = c(1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1))
edges <- data.frame(
from = c(9, 1, 1, 1, 2, 10, 3, 1, 3, 3, 11, 7, 5, 8, 7, 3, 4),
to = c(1, 9, 2, 3, 4, 4, 5, 7, 7, 6, 6, 8, 8, 4, 6, 4, 9),
label = c(50, 10, 50, 50, 30, 15, 50, 20, 40, 30, 22, 30, 30, 20, 30, 10, 40),
curve = c(5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 10),
arrow = "1.2,1.0")
p <- tda_pl_graph(p, nodes, edges)
plot(p)
7.6.1.2 Control
gio computes ownership shares in a directed valued
graph: for each pair, the direct share A(i,j) and the share
including holdings held through other nodes, Y(i,j).
gd11.cf runs it on gd2b.dat.
Box 220gio (gd11.cf)
gd2b <- read.table(ex("gd2b.dat"), col.names = c("I", "J", "V"))
tda_g_ownership(tda_graph(gd2b, directed = TRUE))$table i node_i node_j a_ij y_ij c iterations
1 1 1 1 0.0 0.0000000 4 4
2 1 1 2 0.5 0.5081285 4 4
3 1 1 3 0.0 0.0812940 4 4
4 1 1 4 0.0 0.1524330 4 4
5 2 2 2 0.0 0.0160000 3 2
6 2 2 3 0.1 0.1600000 3 2
7 2 2 4 0.3 0.3000000 3 2
8 3 3 2 0.1 0.1000000 3 1
9 3 3 3 0.0 0.0160000 3 1
10 3 3 4 0.0 0.0300000 3 1
11 4 4 2 0.0 0.0202000 3 2
12 4 4 3 0.2 0.2020200 3 2
13 4 4 4 0.0 0.0060600 3 2Two graphs on one page; in the second a reciprocal pair of edges is drawn as two curves, and all edges carry arrowheads.
Box 221gd9.cf
p <- tda_ps(xlim = c(0, 11.5), ylim = c(0, 4), width = 80, height = 40)
nodesA <- data.frame(id = 1:4, x = c(0.5, 3, 5, 5), y = c(2, 2, 1, 3),
grey = 0.8)
edgesA <- data.frame(from = c(1, 2, 2, 4), to = c(2, 4, 3, 3),
label = c(0.5, 0.3, 0.1, 0.2), arrow = "1.5,1.0")
p <- tda_pl_graph(p, nodesA, edgesA)
nodesB <- data.frame(id = 1:4, x = c(6.5, 9, 11, 11), y = c(2, 2, 1, 3),
grey = 0.8)
edgesB <- data.frame(from = c(1, 2, 2, 4, 3), to = c(2, 4, 3, 3, 2),
label = c(0.5, 0.3, 0.1, 0.2, 0.1),
curve = c(0, 0, 0, 0, -3), arrow = "1.5,1.0")
p <- tda_pl_graph(p, nodesB, edgesB)
p <- tda_pl_text(p, "Graph A", at = c(0.5, 3))
p <- tda_pl_text(p, "Graph B", at = c(6.5, 3))
plot(p)
7.6.1.3 Measures of Flow Centrality
gfc (tda_g(g, "gfc")) is flow centrality on
gd14.dat (gd41.cf), one record per pair of
nodes.
Box 222gfc (gd41.cf)
gd14 <- read.table(ex("gd14.dat"), col.names = c("I", "J", "V"))
tda_g(tda_graph(gd14, directed = TRUE), "gfc", list(fmt = "4.0"))$table i j node_i node_j f1 f2 f3 f4 f5 f6
1 1 2 1 2 3 -1 -1 -1 -1 -1
2 1 3 1 3 6 -1 3 -1 4 -1
3 1 4 1 4 2 -1 -1 -1 -1 -1
4 1 5 1 5 2 -1 2 0 2 -1
5 2 3 2 3 3 -1 -1 -1 -1 -1
6 2 5 2 5 2 -1 -1 0 -1 -1
7 3 5 3 5 2 -1 -1 -1 -1 -1
8 4 3 4 3 2 -1 -1 -1 -1 -1
9 4 5 4 5 2 -1 -1 0 -1 -1The section’s Figure 1 is the directed graph the examples use, drawn
by gd40.cf.
Box 223gd40.cf
p <- tda_ps(xlim = c(0, 8), ylim = c(-1.5, 7), width = 80, height = 40)
nodes <- data.frame(id = 1:5, x = c(1, 3, 5, 3, 7), y = c(3, 6, 3, 0, 3),
grey = 0.8)
edges <- data.frame(
from = c(1, 1, 1, 2, 4, 3),
to = c(2, 3, 4, 3, 3, 5),
label = c(3, 1, 2, 3, 2, 2),
arrow = "1.5,1.0")
p <- tda_pl_graph(p, nodes, edges)
plot(p)
8 Set-valued Data
8.3.2 Evaluation of Inclusion Functions
Chapter 8 is about set-valued data: a case is not a number but an
interval, and the answer is an interval too. tda_evalfi()
evaluates a function over interval arguments.
The manual’s box is evalfi1: the function’s inclusion
over the interval and its derivative’s,
derivative = TRUE.
Box 224An inclusion function and its derivative (evalfi1)
tda_evalfi("sin(x)", x = c(0, 1)) lower upper
0.000000 0.841471 tda_evalfi("sin(x)", x = c(0, 1), derivative = TRUE) lower upper
0.5403023 1.0000000 8.4.1 The gmin Command
tda_gmin() finds the global minimum by subdivision.
start= gives the search interval, as
c(step, lower, upper); tol_fe= is the
tolerance on the function value within which a box is finally accepted.
TDA 6.4p’s default is 1e-10, and with it the box containing the minimum
of sin(x) on [0, 10] is only temporarily accepted (its lower bound is -1
to sixteen digits, but not to ten); 1e-6 accepts it, giving one box with
x in [4.71237, 4.71245].
The manual’s box reports 39 iterations, 41 function evaluations and 77 inclusion evaluations. The program does not: running the manual’s command file unchanged through TDA gives 19, 20 and 35, and so does the call here. The manual is out of date on this box.
Box 225gmin
g <- tda_gmin("sin(x)", start = list(c(1, 0, 10)), use_derivatives = TRUE,
tol_fe = 1e-6)
g$value[1] -1g$boxes box accepted width lower_f upper_f
1 1 2 7.629395e-05 -1 -1g$estimates box parameter lower upper
1 1 x 4.712372 4.712448# the counters TDA prints above the boxes
g$counts iterations evaluations inclusion_evaluations boxes_used
19 20 35 3
boxes_temporary boxes_accepted
1 1 8.5.1 Set-valued Discrete Variables
self_consistent = TRUE asks for the self-consistent
estimate rather than the bounds alone.
Box 227Distribution of a set-valued discrete variable
# the manual's box shows both data files as well as the results
id1 <- read.table(ex("id1.dat"))
id1 V1 V2 V3
1 1 0 0
2 0 1 0
3 0 0 1
4 1 1 0
5 0 1 1tda_sddf(id1, self_consistent = TRUE)Call: tda_sddf(sets = id1, self_consistent = TRUE)
Cases:
category lower upper mean_df self_consistent_df
V1 0.2 0.4 0.3 0.2765957
V2 0.2 0.6 0.4 0.4468085
V3 0.2 0.4 0.3 0.2765957id1a <- read.table(ex("id1a.dat"))
id1a V1 V2
1 1 1
2 2 2
3 3 3
4 1 2
5 2 3tda_iddf(~iv(V1, V2), data = id1a, self_consistent = TRUE)Call: tda_iddf(~iv(V1, V2), data = id1a, self_consistent = TRUE)
Cases: 5
index lower upper mean_df self_consistent_df
1 0.2 0.4 0.3 0.2765957
2 0.2 0.6 0.4 0.4468085
3 0.2 0.4 0.3 0.27659578.5.2 Interval-valued Variables
iv(lower, upper) is the interval-valued variable, the
counterpart of Surv() for this chapter.
Box 228Distribution of an interval-valued variable
id2 <- read.table(ex("id2.dat"))
id2$XL <- id2$V1 - 30L
id2$XH <- id2$V1 + 30L
tda_idf(~iv(XL, XH), data = id2)Call: tda_idf(~iv(XL, XH), data = id2)
Cases: 11
index partition lower upper mean_df
1 1870 0.00000000 0.09090909 0.00000000
2 1920 0.00000000 0.18181818 0.07575758
3 1930 0.09090909 0.18181818 0.10606061
4 1970 0.09090909 0.27272727 0.16666667
5 1980 0.18181818 0.36363636 0.19696970
6 2000 0.18181818 0.45454545 0.25757576
7 2020 0.18181818 0.54545455 0.34848485
8 2030 0.27272727 0.54545455 0.40909091
9 2040 0.36363636 0.54545455 0.45454545
10 2060 0.45454545 0.54545455 0.51515152
11 2070 0.45454545 0.63636364 0.53030303
12 2080 0.54545455 0.72727273 0.56060606
13 2110 0.54545455 0.90909091 0.65151515
14 2120 0.54545455 1.00000000 0.71212121
15 2130 0.63636364 1.00000000 0.78787879
16 2140 0.72727273 1.00000000 0.84848485
17 2170 0.90909091 1.00000000 0.98484848
18 2180 1.00000000 1.00000000 1.000000008.6.1 Mean Values
The mean of an interval-valued variable is itself an interval: every value in it is the mean of some admissible data set.
Box 229imean
tda_imean(~iv(XL, XH), data = id2)Call: tda_imean(~iv(XL, XH), data = id2)
Cases: 11
Bounds: [2022.727, 2082.727]8.6.2 Variances
Box 230ivariance
tda_ivariance(~iv(V1, V2), data = id1a)Call: tda_ivariance(~iv(V1, V2), data = id1a)
Cases: 5
Bounds: [0.4, 0.8]