[PATCH v3] selftests: ftrace: Allow some tests to be run in a tracing instance

[Steven Rostedt]

From 4464dc867ead3ea14654165ad3ab68263aff7b17 Mon Sep 17 00:00:00 2001
From: "Steven Rostedt (VMware)" <***@goodmis.org>
Date: Thu, 20 Apr 2017 12:53:18 -0400
Subject: [PATCH] selftests: ftrace: Allow some tests to be run in a tracing
instance

An tracing instance has several of the same capabilities as the top level
instance, but may be implemented slightly different. Instead of just writing
tests that duplicat the same test cases of the top level instance, allow a
test to be written for both the top level as well as for an instance.

If a test case can be run in both the top level as well as in an tracing
instance directory, then it should add a tag "# flags: instance" in the
header of the test file. Then after all tests have run, any test that has an
instance flag set, will run again within a tracing instance.

Cc: Shuah Khan <***@kernel.org>
Cc: Namhyung Kim <***@kernel.org>
Suggestions-from: Masami Hiramatsu <***@kernel.org>
Signed-off-by: Steven Rostedt (VMware) <***@goodmis.org>
---
tools/testing/selftests/ftrace/ftracetest | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)

diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index a8631d9..3215a8d 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -157,6 +157,10 @@ testcase() { # testfile
prlog -n "[$CASENO]$desc"
}

+test_on_instance() { # testfile
+ grep -q "^#[ \t]*flags:.*instance" $1
+}
+
eval_result() { # sigval
case $1 in
$PASS)
@@ -271,6 +275,21 @@ for t in $TEST_CASES; do
run_test $t
done

+# Test on instance loop
+FIRST_INSTANCE=0
+for t in $TEST_CASES; do
+ test_on_instance $t || continue
+ if [ $FIRST_INSTANCE -eq 0 ]; then
+ FIRST_INSTANCE=1
+ echo "Running tests in a tracing instance:"
+ fi
+ SAVED_TRACING_DIR=$TRACING_DIR
+ export TRACING_DIR=`mktemp -d $TRACING_DIR/instances/ftracetest.XXXXXX`
+ run_test $t
+ rmdir $TRACING_DIR
+ TRACING_DIR=$SAVED_TRACING_DIR
+done
+
prlog ""
prlog "# of passed: " `echo $PASSED_CASES | wc -w`
prlog "# of failed: " `echo $FAILED_CASES | wc -w`

[Masami Hiramatsu]

On Fri, 21 Apr 2017 23:38:50 -0400

Post by Steven Rostedt
From 4464dc867ead3ea14654165ad3ab68263aff7b17 Mon Sep 17 00:00:00 2001
Date: Thu, 20 Apr 2017 12:53:18 -0400
Subject: [PATCH] selftests: ftrace: Allow some tests to be run in a tracing
instance
An tracing instance has several of the same capabilities as the top level
instance, but may be implemented slightly different. Instead of just writing
tests that duplicat the same test cases of the top level instance, allow a
test to be written for both the top level as well as for an instance.
If a test case can be run in both the top level as well as in an tracing
instance directory, then it should add a tag "# flags: instance" in the
header of the test file. Then after all tests have run, any test that has an
instance flag set, will run again within a tracing instance.
---
tools/testing/selftests/ftrace/ftracetest | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index a8631d9..3215a8d 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -157,6 +157,10 @@ testcase() { # testfile
prlog -n "[$CASENO]$desc"
}
+test_on_instance() { # testfile
+ grep -q "^#[ \t]*flags:.*instance" $1

Oops, this needs '&> /dev/null' at the end to suppress its result.

Post by Steven Rostedt
+}
+
eval_result() { # sigval
case $1 in
$PASS)
@@ -271,6 +275,21 @@ for t in $TEST_CASES; do
run_test $t
done
+# Test on instance loop
+FIRST_INSTANCE=0
+for t in $TEST_CASES; do
+ test_on_instance $t || continue
+ if [ $FIRST_INSTANCE -eq 0 ]; then
+ FIRST_INSTANCE=1
+ echo "Running tests in a tracing instance:"
+ fi

Ah, I see. This is important. And I would rather like to show
it on the description line of each test so that we can check
which test log is run in an instance. E.g. passing "(instance)"
message to run_test() and testcase() as the 2nd arg, and print
it in testlog and console?

Thank you,

Post by Steven Rostedt
+ SAVED_TRACING_DIR=$TRACING_DIR
+ export TRACING_DIR=`mktemp -d $TRACING_DIR/instances/ftracetest.XXXXXX`
+ run_test $t
+ rmdir $TRACING_DIR
+ TRACING_DIR=$SAVED_TRACING_DIR
+done
+
prlog ""
prlog "# of passed: " `echo $PASSED_CASES | wc -w`
prlog "# of failed: " `echo $FAILED_CASES | wc -w`
--
2.9.3

[Steven Rostedt]

On Sat, 22 Apr 2017 17:41:49 +0900

Post by Masami Hiramatsu
On Fri, 21 Apr 2017 23:38:50 -0400

Post by Steven Rostedt
From 4464dc867ead3ea14654165ad3ab68263aff7b17 Mon Sep 17 00:00:00 2001
Date: Thu, 20 Apr 2017 12:53:18 -0400
Subject: [PATCH] selftests: ftrace: Allow some tests to be run in a tracing
instance
An tracing instance has several of the same capabilities as the top level
instance, but may be implemented slightly different. Instead of just writing
tests that duplicat the same test cases of the top level instance, allow a
test to be written for both the top level as well as for an instance.
If a test case can be run in both the top level as well as in an tracing
instance directory, then it should add a tag "# flags: instance" in the
header of the test file. Then after all tests have run, any test that has an
instance flag set, will run again within a tracing instance.
---
tools/testing/selftests/ftrace/ftracetest | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index a8631d9..3215a8d 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -157,6 +157,10 @@ testcase() { # testfile
prlog -n "[$CASENO]$desc"
}
+test_on_instance() { # testfile
+ grep -q "^#[ \t]*flags:.*instance" $1

Oops, this needs '&> /dev/null' at the end to suppress its result.

The -q should keep grep from outputting anything. It does on my end. Do
you see something different?

Post by Masami Hiramatsu

Post by Steven Rostedt
+}
+
eval_result() { # sigval
case $1 in
$PASS)
@@ -271,6 +275,21 @@ for t in $TEST_CASES; do
run_test $t
done
+# Test on instance loop
+FIRST_INSTANCE=0
+for t in $TEST_CASES; do
+ test_on_instance $t || continue
+ if [ $FIRST_INSTANCE -eq 0 ]; then
+ FIRST_INSTANCE=1
+ echo "Running tests in a tracing instance:"
+ fi

Ah, I see. This is important. And I would rather like to show
it on the description line of each test so that we can check
which test log is run in an instance. E.g. passing "(instance)"
message to run_test() and testcase() as the 2nd arg, and print
it in testlog and console?

OK, I'll update to v4.

Thanks,

-- Steve

Post by Masami Hiramatsu
Thank you,

Post by Steven Rostedt
+ SAVED_TRACING_DIR=$TRACING_DIR
+ export TRACING_DIR=`mktemp -d $TRACING_DIR/instances/ftracetest.XXXXXX`
+ run_test $t
+ rmdir $TRACING_DIR
+ TRACING_DIR=$SAVED_TRACING_DIR
+done
+
prlog ""
prlog "# of passed: " `echo $PASSED_CASES | wc -w`
prlog "# of failed: " `echo $FAILED_CASES | wc -w`
--
2.9.3

[Masami Hiramatsu]

On Sat, 22 Apr 2017 06:58:37 -0400

Post by Steven Rostedt
On Sat, 22 Apr 2017 17:41:49 +0900

Post by Masami Hiramatsu
On Fri, 21 Apr 2017 23:38:50 -0400

Post by Steven Rostedt
From 4464dc867ead3ea14654165ad3ab68263aff7b17 Mon Sep 17 00:00:00 2001
Date: Thu, 20 Apr 2017 12:53:18 -0400
Subject: [PATCH] selftests: ftrace: Allow some tests to be run in a tracing
instance
An tracing instance has several of the same capabilities as the top level
instance, but may be implemented slightly different. Instead of just writing
tests that duplicat the same test cases of the top level instance, allow a
test to be written for both the top level as well as for an instance.
If a test case can be run in both the top level as well as in an tracing
instance directory, then it should add a tag "# flags: instance" in the
header of the test file. Then after all tests have run, any test that has an
instance flag set, will run again within a tracing instance.
---
tools/testing/selftests/ftrace/ftracetest | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index a8631d9..3215a8d 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -157,6 +157,10 @@ testcase() { # testfile
prlog -n "[$CASENO]$desc"
}
+test_on_instance() { # testfile
+ grep -q "^#[ \t]*flags:.*instance" $1

Oops, this needs '&> /dev/null' at the end to suppress its result.

The -q should keep grep from outputting anything. It does on my end. Do
you see something different?

Ah, sorry I missed -q option... OK, it is good!

Post by Steven Rostedt

Post by Masami Hiramatsu

Post by Steven Rostedt
+}
+
eval_result() { # sigval
case $1 in
$PASS)
@@ -271,6 +275,21 @@ for t in $TEST_CASES; do
run_test $t
done
+# Test on instance loop
+FIRST_INSTANCE=0
+for t in $TEST_CASES; do
+ test_on_instance $t || continue
+ if [ $FIRST_INSTANCE -eq 0 ]; then
+ FIRST_INSTANCE=1
+ echo "Running tests in a tracing instance:"
+ fi

Ah, I see. This is important. And I would rather like to show
it on the description line of each test so that we can check
which test log is run in an instance. E.g. passing "(instance)"
message to run_test() and testcase() as the 2nd arg, and print
it in testlog and console?

OK, I'll update to v4.

Thank you!

Post by Steven Rostedt
Thanks,
-- Steve

Post by Masami Hiramatsu
Thank you,

Post by Steven Rostedt
+ SAVED_TRACING_DIR=$TRACING_DIR
+ export TRACING_DIR=`mktemp -d $TRACING_DIR/instances/ftracetest.XXXXXX`
+ run_test $t
+ rmdir $TRACING_DIR
+ TRACING_DIR=$SAVED_TRACING_DIR
+done
+
prlog ""
prlog "# of passed: " `echo $PASSED_CASES | wc -w`
prlog "# of failed: " `echo $FAILED_CASES | wc -w`
--
2.9.3

[Steven Rostedt]

On Sun, 23 Apr 2017 09:02:24 +0900

Post by Steven Rostedt

Post by Masami Hiramatsu

Post by Steven Rostedt
+}
+
eval_result() { # sigval
case $1 in
$PASS)
@@ -271,6 +275,21 @@ for t in $TEST_CASES; do
run_test $t
done
+# Test on instance loop
+FIRST_INSTANCE=0
+for t in $TEST_CASES; do
+ test_on_instance $t || continue
+ if [ $FIRST_INSTANCE -eq 0 ]; then
+ FIRST_INSTANCE=1
+ echo "Running tests in a tracing instance:"
+ fi

Ah, I see. This is important. And I would rather like to show
it on the description line of each test so that we can check
which test log is run in an instance. E.g. passing "(instance)"
message to run_test() and testcase() as the 2nd arg, and print
it in testlog and console?

OK, I'll update to v4.

What about something like this:

-- Steve

diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index a8631d9..32e6211 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -150,11 +150,16 @@ XFAILED_CASES=
UNDEFINED_CASES=
TOTAL_RESULT=0

+INSTANCE=
CASENO=0
testcase() { # testfile
CASENO=$((CASENO+1))
desc=`grep "^#[ \t]*description:" $1 | cut -f2 -d:`
- prlog -n "[$CASENO]$desc"
+ prlog -n "[$CASENO]$INSTANCE$desc"
+}
+
+test_on_instance() { # testfile
+ grep -q "^#[ \t]*flags:.*instance" $1
}

eval_result() { # sigval
@@ -271,6 +276,17 @@ for t in $TEST_CASES; do
run_test $t
done

+# Test on instance loop
+INSTANCE=" (instance) "
+for t in $TEST_CASES; do
+ test_on_instance $t || continue
+ SAVED_TRACING_DIR=$TRACING_DIR
+ export TRACING_DIR=`mktemp -d $TRACING_DIR/instances/ftracetest.XXXXXX`
+ run_test $t
+ rmdir $TRACING_DIR
+ TRACING_DIR=$SAVED_TRACING_DIR
+done
+
prlog ""
prlog "# of passed: " `echo $PASSED_CASES | wc -w`
prlog "# of failed: " `echo $FAILED_CASES | wc -w`

[Masami Hiramatsu]

On Mon, 24 Apr 2017 18:01:38 -0400

Post by Steven Rostedt
On Sun, 23 Apr 2017 09:02:24 +0900

Post by Steven Rostedt

Post by Masami Hiramatsu

Post by Steven Rostedt
+}
+
eval_result() { # sigval
case $1 in
$PASS)
@@ -271,6 +275,21 @@ for t in $TEST_CASES; do
run_test $t
done
+# Test on instance loop
+FIRST_INSTANCE=0
+for t in $TEST_CASES; do
+ test_on_instance $t || continue
+ if [ $FIRST_INSTANCE -eq 0 ]; then
+ FIRST_INSTANCE=1
+ echo "Running tests in a tracing instance:"
+ fi

Ah, I see. This is important. And I would rather like to show
it on the description line of each test so that we can check
which test log is run in an instance. E.g. passing "(instance)"
message to run_test() and testcase() as the 2nd arg, and print
it in testlog and console?

OK, I'll update to v4.

Ah, that's much better than I thought :)

Acked-by: Masami Hiramatsu <***@kernel.org>

Thank you!

Post by Steven Rostedt
-- Steve
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index a8631d9..32e6211 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -150,11 +150,16 @@ XFAILED_CASES=
UNDEFINED_CASES=
TOTAL_RESULT=0
+INSTANCE=
CASENO=0
testcase() { # testfile
CASENO=$((CASENO+1))
desc=`grep "^#[ \t]*description:" $1 | cut -f2 -d:`
- prlog -n "[$CASENO]$desc"
+ prlog -n "[$CASENO]$INSTANCE$desc"
+}
+
+test_on_instance() { # testfile
+ grep -q "^#[ \t]*flags:.*instance" $1
}
eval_result() { # sigval
@@ -271,6 +276,17 @@ for t in $TEST_CASES; do
run_test $t
done
+# Test on instance loop
+INSTANCE=" (instance) "
+for t in $TEST_CASES; do
+ test_on_instance $t || continue
+ SAVED_TRACING_DIR=$TRACING_DIR
+ export TRACING_DIR=`mktemp -d $TRACING_DIR/instances/ftracetest.XXXXXX`
+ run_test $t
+ rmdir $TRACING_DIR
+ TRACING_DIR=$SAVED_TRACING_DIR
+done
+
prlog ""
prlog "# of passed: " `echo $PASSED_CASES | wc -w`
prlog "# of failed: " `echo $FAILED_CASES | wc -w`