Discussion:
RFC: How to handle function tracing, frame pointers and -mfentry?
(too old to reply)
Steven Rostedt
2012-04-27 18:50:01 UTC
Permalink
I have code that implements -mfentry for the function tracer.

For those that are unfamiliar with this, it was introduced into gcc in
4.6.0 for x86 only. Used with the -pg option, it replaces mcount with
fentry, and is placed at the first instruction of a function (instead of
after the frame has been built, as mcount does).

So what's my problem? With -mfentry, function tracer does not depend on
frame pointers. As frame pointers may add overhead, some distros
(Debian) has disabled frame pointers from their kernels and with that,
has also disabled function tracing.

Currently, function tracing selects CONFIG_FRAME_POINTER for various
archs (including x86), as the kernel will not compile without it, if
function tracing is enabled. But if -mfentry is available with the
compiler, it does not have this dependency. The kernel will compile fine
with -pg -mfentry and without frame pointers.

My question is, how do I remove the dependency in kconfig based on the
compiler?

I've tried a few things and here's some ideas:

1) Have kconfig detect if -mfentry is supported with the current
compiler. If it is, then enable a "auto" config called
CONFIG_CC_HAS_FENTRY, and allow function tracer be able to select
FRAME_POINTER if that's not defined.

I actually got this to work, but it only works if the host compiler is
the same as the compiler building the kernel. Which in lots of cases is
not (my default setup does not have this).

2) Remove the select entirely. On build, if the compiler fails because
it does not support -pg running without frame pointers, have a big error
message telling the user they can't have function tracing without frame
pointers and that they either need to enable frame pointers or disable
function tracing.

3) Add frame pointers silently if gcc fails to build with gcc -pg.

I personally do not really care which approach is taken (or a new
approach that I didn't list). I just want to be able to build a kernel
with function tracing and without frame pointers on x86.

Thanks,

-- Steve


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Steven Rostedt
2012-04-27 19:50:01 UTC
Permalink
Post by Steven Rostedt
Currently, function tracing selects CONFIG_FRAME_POINTER for various
archs (including x86), as the kernel will not compile without it, if
function tracing is enabled. But if -mfentry is available with the
compiler, it does not have this dependency. The kernel will compile fine
with -pg -mfentry and without frame pointers.
Just to show the difference. I ran several iterations of hackbench, on
an Intel Quad Core2 2.6GHz.

Here's the fentry code with frame pointers (tracing disabled):

Time: 2.006
Time: 2.028
Time: 2.028
Time: 1.999
Time: 2.035
Time: 2.037
Time: 2.006
Time: 1.996
Time: 2.049
Time: 1.991
Time: 2.038
Time: 2.047
Time: 2.039
Time: 2.000
Time: 2.021
Time: 2.011
Time: 2.007
Time: 2.024
Time: 2.033
Time: 2.027
Time: 2.044


And the fentry code with frame pointers disabled:

Time: 1.870
Time: 1.861
Time: 1.865
Time: 1.884
Time: 1.867
Time: 1.867
Time: 1.875
Time: 1.883
Time: 1.863
Time: 1.877
Time: 1.865
Time: 1.885
Time: 1.842
Time: 1.863
Time: 1.899
Time: 1.877
Time: 1.837
Time: 1.900
Time: 1.897
Time: 1.877
Time: 1.853
Time: 1.856

That's about a 8% difference.

-- Steve


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Sam Ravnborg
2012-04-27 20:30:02 UTC
Permalink
Post by Steven Rostedt
I have code that implements -mfentry for the function tracer.
For those that are unfamiliar with this, it was introduced into gcc in
4.6.0 for x86 only. Used with the -pg option, it replaces mcount with
fentry, and is placed at the first instruction of a function (instead of
after the frame has been built, as mcount does).
So what's my problem? With -mfentry, function tracer does not depend on
frame pointers. As frame pointers may add overhead, some distros
(Debian) has disabled frame pointers from their kernels and with that,
has also disabled function tracing.
Currently, function tracing selects CONFIG_FRAME_POINTER for various
archs (including x86), as the kernel will not compile without it, if
function tracing is enabled. But if -mfentry is available with the
compiler, it does not have this dependency. The kernel will compile fine
with -pg -mfentry and without frame pointers.
My question is, how do I remove the dependency in kconfig based on the
compiler?
Allow the user to select one of:
1) function tracer
2) frame pointer
3) none of the above

This should be trivial to do in Kconfig language.

If "function tracer" is selected then use -mfentry if supported,
with a fallback to frame pointers.

This looks simple IMO and should be easy to implement too.

Sam
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
H. Peter Anvin
2012-04-27 20:40:02 UTC
Permalink
Post by Sam Ravnborg
Post by Steven Rostedt
My question is, how do I remove the dependency in kconfig based on the
compiler?
1) function tracer
2) frame pointer
3) none of the above
This should be trivial to do in Kconfig language.
If "function tracer" is selected then use -mfentry if supported,
with a fallback to frame pointers.
This looks simple IMO and should be easy to implement too.
Except it is the wrong thing. This is not the only user of frame
pointer. What I think you mean is remove the dependency in Kconfig, but
force the frame pointer enabled if -mfentry is not supported.

-hpa


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Steven Rostedt
2012-04-27 21:00:04 UTC
Permalink
Post by H. Peter Anvin
Post by Sam Ravnborg
Post by Steven Rostedt
My question is, how do I remove the dependency in kconfig based on the
compiler?
1) function tracer
2) frame pointer
3) none of the above
This should be trivial to do in Kconfig language.
If "function tracer" is selected then use -mfentry if supported,
with a fallback to frame pointers.
This looks simple IMO and should be easy to implement too.
Except it is the wrong thing. This is not the only user of frame
pointer. What I think you mean is remove the dependency in Kconfig, but
force the frame pointer enabled if -mfentry is not supported.
You mean option 3?

3) Add frame pointers silently if gcc fails to build with gcc -pg.

Which I should have added, relies on the same thing as option 2:

2) Remove the select entirely.


-- Steve


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
H. Peter Anvin
2012-04-27 21:20:02 UTC
Permalink
Post by Steven Rostedt
Post by H. Peter Anvin
Except it is the wrong thing. This is not the only user of frame
pointer. What I think you mean is remove the dependency in Kconfig, but
force the frame pointer enabled if -mfentry is not supported.
You mean option 3?
3) Add frame pointers silently if gcc fails to build with gcc -pg.
Yes, your option 3.

-hypa

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Steven Rostedt
2012-04-27 21:30:02 UTC
Permalink
Post by H. Peter Anvin
Post by Steven Rostedt
Post by H. Peter Anvin
Except it is the wrong thing. This is not the only user of frame
pointer. What I think you mean is remove the dependency in Kconfig, but
force the frame pointer enabled if -mfentry is not supported.
You mean option 3?
3) Add frame pointers silently if gcc fails to build with gcc -pg.
Yes, your option 3.
Actually, I just noticed this in the Makefile:

ifdef CONFIG_FRAME_POINTER
KBUILD_CFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls
else
# Some targets (ARM with Thumb2, for example), can't be built with frame
# pointers. For those, we don't have FUNCTION_TRACER automatically
# select FRAME_POINTER. However, FUNCTION_TRACER adds -pg, and this is
# incompatible with -fomit-frame-pointer with current GCC, so we don't use
# -fomit-frame-pointer with FUNCTION_TRACER.
ifndef CONFIG_FUNCTION_TRACER
KBUILD_CFLAGS += -fomit-frame-pointer
endif
endif


Because of ARM thumb2, -fomit-frame-pointer is not added if
CONFIG_FRAME_POINTER is not set and FUNCTION_TRACER is. As it is fine to
just use '-pg', which gcc will add frame pointers if needed, and it is
only a problem if both -pg and -fomit-frame-pointer is set. We only need
to remove the select without doing anything else.

If -mfentry is added with -pg, and -fno-omit-frame-pointer is not set,
gcc will not add frame pointers by default.

So it seems that gcc took care of this problem for me :-)

I'll go and make a patch that simply removes the select.

Sorry for the noise ;-)

-- Steve


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Andi Kleen
2012-04-28 08:40:01 UTC
Permalink
Post by Steven Rostedt
1) Have kconfig detect if -mfentry is supported with the current
compiler. If it is, then enable a "auto" config called
CONFIG_CC_HAS_FENTRY, and allow function tracer be able to select
FRAME_POINTER if that's not defined.
I actually got this to work, but it only works if the host compiler is
the same as the compiler building the kernel. Which in lots of cases is
not (my default setup does not have this).
Kconfig just needs to learn how to run the target compiler

I think that's the right direction. Right now our main
Makefiles get polluted more and more with "test compiles", each
of which makes a "null make" slower and slower.

I just measured and a null compile (nothing changes) of a current
tree calls "gcc" 141 times.

All this stuff should be cached in the Kconfig instead.

It may break some obscure setups (that can be probably fixed without
too much effort), but the development turnaround improvement
for everyone else would be worth it.

-Andi
--
***@linux.intel.com -- Speaking for myself only.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Sam Ravnborg
2012-04-28 09:00:02 UTC
Permalink
Post by Andi Kleen
I just measured and a null compile (nothing changes) of a current
tree calls "gcc" 141 times.
Something is wrong then which must be fixed.
There are 55 uses of cc-option in */x86/*
And only 7 uses of cc-option in the top-level Makefile.

So it looks like we evalute the same assignmet twice or more.

Sam
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Sam Ravnborg
2012-04-28 09:20:02 UTC
Permalink
Post by Sam Ravnborg
Post by Andi Kleen
I just measured and a null compile (nothing changes) of a current
tree calls "gcc" 141 times.
Something is wrong then which must be fixed.
There are 55 uses of cc-option in */x86/*
And only 7 uses of cc-option in the top-level Makefile.
So it looks like we evalute the same assignmet twice or more.
"make defconfig" in my box calls gcc 50 times.
This looks much closer to the actual usage of cc-option in the
above mentioned files.
How did you manage to achieve 144 calls to gcc?

Any specific configuration that triggers this?

Sam
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Andi Kleen
2012-04-28 12:20:01 UTC
Permalink
Post by Sam Ravnborg
"make defconfig" in my box calls gcc 50 times.
This looks much closer to the actual usage of cc-option in the
above mentioned files.
How did you manage to achieve 144 calls to gcc?
Any specific configuration that triggers this?
Hmm, thanks for double checking. I suspect I misgrepped
(included the children of the gcc from the strace log).
If it calls two children that would be roughly your number.

But even 50 is quite bad.

-Andi
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
H. Peter Anvin
2012-04-28 16:40:01 UTC
Permalink
Post by Andi Kleen
Kconfig just needs to learn how to run the target compiler
I think that's the right direction. Right now our main
Makefiles get polluted more and more with "test compiles", each
of which makes a "null make" slower and slower.
I just measured and a null compile (nothing changes) of a current
tree calls "gcc" 141 times.
All this stuff should be cached in the Kconfig instead.
It may break some obscure setups (that can be probably fixed without
too much effort), but the development turnaround improvement
for everyone else would be worth it.
I don't think it is a performance issue as much as a utility issue.
When Kconfig can't actually reflect what is built into the resulting
kernel, that is a problem.

-hpa
--
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel. I don't speak on their behalf.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Loading...