The goal is to develop a general purpose, programmable, application monitor which will display information on the progress of a running application.
Client: Prof. David SaundersSome programs put on a sideshow. If you've ever downloaded a software package, done a software update, or watched a video, you know what I mean. The tool manages to display some blinking lights, or perhaps a box with a colored bar growing from left to right as the task progresses, or even sometimes a frequently updated "percentage done" number. With blinking lights you at least know the tool is still running, not hopelessly stalled. With percentage-done displays you know more, but not that much perhaps: the growth in percent done tends to be extremely erratic. It's not surprising; the rate of data transfer depends on many things and future rate of data flow is hard to predict.
Also, the developer effort put into this sideshow may be minimal. After all, the programmer's main task is to complete the update or buffering or whatever, not to precisely determine how long it will take. Sometimes, we've seen a completion bar fill up only to start over again from zero. This indicated that a sideshow has been arranged for some subtasks, but no one has integrated it for the top level task.
Bottom line: Whenever a program is going to take a while to complete, (1) it is helpful to see some side information about how it's going along the way, but (2) it is hard for the programmer to provide this, though even an imperfect sideshow is better than nothing.
Data transfers are easy. Let's take this to the next level, to long running scientific codes. If my computation may take hours or days, it would be nice to have some indication of the stage the computation has reached and some partial results along the way.
Example: Say you want to solve a gigantic system of linear equations.
Shortly after starting the run you see: Started factoring the matrix..
A minute later: Completed 100 of 30000 pivot steps.
...
An hour later: Factorization half done.
...
An hour after that: Rank of the system is 28901
Then: Factorization complete, it took xxx seconds.
Then in rapid succession: First triangular solve complete,
..., Second triangular solve complete,
Solution written to file xyz, and program terminates.
So that is the main idea, to have a system that will provide some useful information as a sideshow to the main event. There are a number of issues to cope with in designing such a system. For instance, the example above may have generated a few dozen messages during the course of a couple of hours. However another application may solve thousands of small linear systems instead of one large one. In that case, a sideshow consisting of thousands of dozens of messages (dozens for each system solved) will be overwhelming and not so helpful. It will be wise to throttle the output from routines deeply nested in the call stack. They will be too far from the user's primary concerns and less likely to inform constructively. We want a system that meets these goals:
There is a precedent to the sideshow idea that has been used in the C++ library LinBox. It is called the commentator. This sideshow project is motivated in large part by the desire to overcome the deficiencies of the commentator and produce a better tool. In hindsight, the commentator is too complex. The programmer has to make decisions about level of importance and deal with a system of message categories that is just too complex to bother with. As it has evolved, the commentator gets wide use in the test suite, but very little use in the library functions themselves. We want a system that will work in the library and serve the user, not just the tester.
Sideshow has two distinct components, the programmer interface and the user interface. The sideshow commentary generated by a running program using the programmer interface will be communicated to the user side either by way of a file or through a socket. The heart of the project is a good user iterface to enable filtering of the stream of commentary so as to meet the user's needs.
The programmer interface will be very "bare bones" and we will want
one for each supported programming language. For the project, a C++
interface will suffice. Sideshow can be an output stream (ofstream)
and the function can be the output operator, thusly: sideshow \<\<
"Computing determinant" \<\< endl; Each message will be tagged
with the time and call stack depth. Some messages will be multiline.
Perhaps we can use \n for non-message-end end of line and
endl for end of message.
The user interface can be written in your language of choice (e.g., Java) and will be responsible for creating a window in which filtered messages are displayed and in which the user can control the filtering, controlling the frequency of messages shown the categories of messages shown, and details about the display (depth of message via indentation? Summary of elapsed subtask time determined by groups of messages?) . Call stack information will be used to help with this process. Among other things, call stack depth can be used to overcome programmer lapses.
For example, if a message is "Starting X", we expect a later message saying "Done with X". However, if a message from higher on the call stack appears before a corresponding "Done" message, we will infer that task X is done.
Here is an example of commentator output. We can do a lot better!
saunders@hmrg:~/software/linbox/tests> test-transpose rep
Writing report data to rep
transpose black box test suite...
test on ScalarMatrix...
Testing Transpose...
Iteration 0...
testLinearity pass (-2.49875e-19 s)
Iteration 0...
testTranspose pass (-2.49875e-19 s)
passed (-2.49875e-19 s)
passed (-2.49875e-19 s)
test on BlasMatrix...
Testing Transpose...
Iteration 0...
testLinearity pass (-2.49875e-19 s)
Iteration 0...
testTranspose pass (-2.49875e-19 s)
passed (-2.49875e-19 s)
Testing Transpose...
passed (-2.49875e-19 s)
passed (-2.49875e-19 s)
test on TriplesBB...
passed (-2.49875e-19 s)
test on COO...
Testing Transpose...
Iteration 0...
testLinearity pass (-2.49875e-19 s)
Iteration 0...
testTranspose pass (-2.49875e-19 s)
passed (-2.49875e-19 s)
Testing Transpose...
passed (-2.49875e-19 s)
passed (-2.49875e-19 s)
passed (-2.49875e-19 s)
saunders@hmrg:~/software/linbox/tests> cat rep
$$(0, 1, Internal description)transpose black box test suite
$$(1, 1, Internal description)test on ScalarMatrix
$$(2, 1, Internal description)Testing Transpose
$$(3, 1, Internal description)testBlackbox on 10 by 10 matrix.
Blackbox = transpose of:%%MatrixMarket matrix coordinate integer general
% written by LinBox::ScalarMatrix, field = Modular( 101 )
10 10 1
1 1 5
$$(3, 1, Internal description)Blackbox linearity test [that A.apply to (ax + y) == a A.apply to x + A.apply to y]
$$(3, 1, Internal description)Iteration 0
Input vector x: [1, 78, 32, 36, 6, 50, 42, 4, 91, 47]
Input vector y: [37, 9, 68, 42, 97, 41, 31, 23, 7, 61]
Input alpha: 55
x+alpha y = [16, 68, 35, 23, 89, 83, 30, 57, 72, 69]
A(x+alpha y) = [80, 37, 74, 14, 41, 11, 49, 83, 57, 42]
Ax = [5, 87, 59, 79, 30, 48, 8, 20, 51, 33]
Ay = [84, 45, 37, 8, 81, 3, 54, 14, 35, 2]
Ax+alpha Ay = [80, 37, 74, 14, 41, 11, 49, 83, 57, 42]
$$(3, 1, Internal description)Finished activity (rea: 0.0001051s, cpu: 0s, sys: 0s): testLinearity pass
$$(3, 1, Internal description)Blackbox transpose test [that u^T(Av) == (uA)^T v]
$$(3, 1, Internal description)Iteration 0
Input vector u: [36, 94, 43, 94, 36, 83, 72, 100, 1, 68]
Input vector v: [68, 80, 100, 3, 52, 15, 11, 57, 5, 58]
Result of apply: [37, 97, 96, 15, 58, 75, 55, 83, 25, 88]
Result of transpose apply: [79, 66, 13, 66, 79, 11, 57, 96, 5, 37]
: 49
: 49
$$(3, 1, Internal description)Finished activity (rea: 6.906e-05s, cpu: 0s, sys: 0s): testTranspose pass
$$(2, 1, Internal description)Finished activity (rea: 0.000768s, cpu: 0s, sys: 0s): passed
$$(2, 1, Internal warning)Activity report mismatch. Check that start () and stop () calls are paired correctly.
$$(1, 1, Internal description)Finished activity (rea: 0.000861s, cpu: 0s, sys: 0s): passed
$$(1, 1, Internal description)test on BlasMatrix
$$(2, 1, Internal description)Testing Transpose
$$(3, 1, Internal description)testBlackbox on 10 by 8 matrix.
Blackbox = transpose of:%%MatrixMarket matrix array integer general
% written by LinBox::BlasSubmatrix, field = Modular( 101 )
8 10
0
0
0
0
0
0
0
0
0
1
2
3
4
5
6
7
0
2
4
6
8
10
12
14
0
3
6
9
12
15
18
21
0
4
8
12
16
20
24
28
0
5
10
15
20
25
30
35
0
6
12
18
24
30
36
42
0
7
14
21
28
35
42
49
0
8
16
24
32
40
48
56
0
9
18
27
36
45
54
63
$$(3, 1, Internal description)Blackbox linearity test [that A.apply to (ax + y) == a A.apply to x + A.apply to y]
$$(3, 1, Internal description)Iteration 0
Input vector x: [1, 78, 32, 36, 6, 50, 42, 4]
Input vector y: [91, 47, 37, 9, 68, 42, 97, 41]
Input alpha: 31
x+alpha y = [95, 20, 68, 12, 94, 39, 19, 63]
A(x+alpha y) = [0, 5, 10, 15, 20, 25, 30, 35, 40, 45]
Ax = [0, 97, 93, 89, 85, 81, 77, 73, 69, 65]
Ay = [0, 85, 69, 53, 37, 21, 5, 90, 74, 58]
Ax+alpha Ay = [0, 5, 10, 15, 20, 25, 30, 35, 40, 45]
$$(3, 1, Internal description)Finished activity (rea: 0.000114s, cpu: 0s, sys: 0s): testLinearity pass
$$(3, 1, Internal description)Blackbox transpose test [that u^T(Av) == (uA)^T v]
$$(3, 1, Internal description)Iteration 0
Input vector u: [23, 7, 61, 55, 36, 94, 43, 94, 36, 83]
Input vector v: [72, 100, 1, 68, 68, 80, 100, 3]
Result of apply: [0, 84, 67, 50, 33, 16, 100, 83, 66, 49]
Result of transpose apply: [0, 31, 62, 93, 23, 54, 85, 15]
: 79
: 79
$$(3, 1, Internal description)Finished activity (rea: 8.207e-05s, cpu: 0s, sys: 0s): testTranspose pass
$$(2, 1, Internal description)Finished activity (rea: 0.008502s, cpu: 0s, sys: 0.004s): passed
$$(2, 1, Internal description)Testing Transpose
$$(2, 1, Internal description)Finished activity (rea: 8.975e-06s, cpu: 0s, sys: 0s): passed
$$(2, 1, Internal warning)Activity report mismatch. Check that start () and stop () calls are paired correctly.
$$(1, 1, Internal description)Finished activity (rea: 0.008691s, cpu: 0s, sys: 0.004s): passed
$$(1, 1, Internal description)test on TriplesBB
$$(2, 1, Internal warning)Activity report mismatch. Check that start () and stop () calls are paired correctly.
$$(1, 1, Internal description)Finished activity (rea: 2.911e-06s, cpu: 0s, sys: 0s): passed
$$(1, 1, Internal description)test on COO
$$(2, 1, Internal description)Testing Transpose
$$(3, 1, Internal description)testBlackbox on 10 by 8 matrix.
Blackbox = transpose of:%%MatrixMarket matrix coordinate integer general
% written by LinBox::SparseMatrix, field = Modular( 101 )
8 10 8
1 1 1
2 2 2
3 3 3
4 4 4
5 5 5
6 6 6
7 7 7
8 8 8
$$(3, 1, Internal description)Blackbox linearity test [that A.apply to (ax + y) == a A.apply to x + A.apply to y]
$$(3, 1, Internal description)Iteration 0
Input vector x: [1, 78, 32, 36, 6, 50, 42, 4]
Input vector y: [91, 47, 37, 9, 68, 42, 97, 41]
Input alpha: 31
x+alpha y = [95, 20, 68, 12, 94, 39, 19, 63]
A(x+alpha y) = [95, 40, 2, 48, 66, 32, 32, 100, 0, 0]
Ax = [1, 55, 96, 43, 30, 98, 92, 32, 0, 0]
Ay = [91, 94, 10, 36, 37, 50, 73, 25, 0, 0]
Ax+alpha Ay = [95, 40, 2, 48, 66, 32, 32, 100, 0, 0]
$$(3, 1, Internal description)Finished activity (rea: 0.0001229s, cpu: 0s, sys: 0s): testLinearity pass
$$(3, 1, Internal description)Blackbox transpose test [that u^T(Av) == (uA)^T v]
$$(3, 1, Internal description)Iteration 0
Input vector u: [23, 7, 61, 55, 36, 94, 43, 94, 36, 83]
Input vector v: [72, 100, 1, 68, 68, 80, 100, 3]
Result of apply: [72, 99, 3, 70, 37, 76, 94, 24, 0, 0]
Result of transpose apply: [23, 14, 82, 18, 79, 59, 99, 45]
: 47
: 47
$$(3, 1, Internal description)Finished activity (rea: 6.911e-05s, cpu: 0s, sys: 0s): testTranspose pass
$$(2, 1, Internal description)Finished activity (rea: 0.0007271s, cpu: 0s, sys: 0s): passed
$$(2, 1, Internal description)Testing Transpose
$$(2, 1, Internal description)Finished activity (rea: 9.089e-06s, cpu: 0s, sys: 0s): passed
$$(2, 1, Internal warning)Activity report mismatch. Check that start () and stop () calls are paired correctly.
$$(1, 1, Internal description)Finished activity (rea: 0.000932s, cpu: 0s, sys: 0s): passed
$$(1, 1, Internal warning)Activity report mismatch. Check that start () and stop () calls are paired correctly.
$$(0, 1, Internal description)Finished activity (rea: 0.01089s, cpu: 0s, sys: 0.004s): passed
That's all, Folks!
saunders@hmrg:~/software/linbox/tests>