Discussion:
Forbid deletion of memory mappings
(too old to reply)
Clemens Kolbitsch
2007-08-30 16:50:11 UTC
Permalink
Hi everyone!
I have a strange problem where I don't even know if there is a solution to it
at the moment:

I'm working on a new way of doing memory-management and currently I allocate
memory mappings (at non-fixed locations) in user-memory (i.e. < TASK_SIZE) in
addition to the regular pages mapped for the apps (e.g. heap memory, etc.)

It all works perfectly well (creating & deleting the additional mappings),
however, when the kernel feels like it needs to allocate a mapping in
user-space it sometimes deletes my mapping and overwrites it with the new
one, although there is plenty of free memory at some other location.

typically my mappings are (automatically chosen) located somewhere around
0xb7xxxxxx although there's plenty free space around 0xayyyyyyy. (i know that
this is a bad location because of stack, etc.) but i need the kernel to
choose a good location for me...

is there a way to make sure my mappings are not removed from memory?? or is
there a location where i can put my mappings that they will not be removed
(i.e. by using MAP_FIXED)?
Note that i need a big amount of memory so the range will have to be fairly
big (to be precise: i need a lot of 0x1000byte mappings - they are always
page-sized, not bigger, not smaller, but A LOT of them :-/ )

hope someone can help me with this!
thanks & greets,
Clemens
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Jiri Kosina
2007-08-30 17:10:07 UTC
Permalink
Post by Clemens Kolbitsch
It all works perfectly well (creating & deleting the additional
mappings), however, when the kernel feels like it needs to allocate a
mapping in user-space it sometimes deletes my mapping and overwrites it
with the new one, although there is plenty of free memory at some other
location.
Hi Clemens,

what do you mean by "overwrites it"? It just probably merges your vma with
the newly created one, right?
--
Jiri Kosina
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Clemens Kolbitsch
2007-08-30 17:10:10 UTC
Permalink
Post by Jiri Kosina
Post by Clemens Kolbitsch
It all works perfectly well (creating & deleting the additional
mappings), however, when the kernel feels like it needs to allocate a
mapping in user-space it sometimes deletes my mapping and overwrites it
with the new one, although there is plenty of free memory at some other
location.
Hi Clemens,
what do you mean by "overwrites it"? It just probably merges your vma with
the newly created one, right?
that's exactly the problem... it replaces my mapping with the new one... i.e.
it first deletes my mapping and then creates the new one at the same
location.

i *guess* this all happens in exec.c, lines 1033ff:

munmap_back:
vma = find_vma_prepare(mm, addr, &prev, &rb_link, &rb_parent);
if (vma && vma->vm_start < addr + len) {
if (do_munmap(mm, addr, len))
return -ENOMEM;
goto munmap_back;
}
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Clemens Kolbitsch
2007-08-30 21:40:08 UTC
Permalink
Post by Jiri Kosina
Post by Clemens Kolbitsch
It all works perfectly well (creating & deleting the additional
mappings), however, when the kernel feels like it needs to allocate a
mapping in user-space it sometimes deletes my mapping and overwrites it
with the new one, although there is plenty of free memory at some other
location.
Hi Clemens,
what do you mean by "overwrites it"? It just probably merges your vma with
the newly created one, right?
does really noone have an answer for me?? i'm having the hardest time to find
a work-around for it.

is there no way to tell the kernel, that a certain mapping must not be
removed, no matter what (except of course an explicit call to sys_unmap, of
course)?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Jiri Kosina
2007-08-30 21:40:09 UTC
Permalink
Post by Clemens Kolbitsch
is there no way to tell the kernel, that a certain mapping must not be
removed, no matter what (except of course an explicit call to sys_unmap,
of course)?
I don't seem to get what is the issue here. Your mapping is not removed,
only the VMAs are merged together into one larger VMA if they have
neighbouring address ranges and compatible protection bits. See
vma_merge().
--
Jiri Kosina
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Clemens Kolbitsch
2007-08-30 21:50:04 UTC
Permalink
Post by Jiri Kosina
Post by Clemens Kolbitsch
is there no way to tell the kernel, that a certain mapping must not be
removed, no matter what (except of course an explicit call to sys_unmap,
of course)?
I don't seem to get what is the issue here. Your mapping is not removed,
only the VMAs are merged together into one larger VMA if they have
neighbouring address ranges and compatible protection bits. See
vma_merge().
the thing is that they are not. the kernel chooses to REPLACE my mapping.

consider the user-space code:

mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);

here, the second call to mmap will shorten the first mapping to 0x1000 bytes
and create one big vma with size 0x5000 bytes.

is there a way to tell it that the second mmap MUST fail?


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
V***@vt.edu
2007-08-30 22:00:11 UTC
Permalink
Post by Clemens Kolbitsch
Post by Jiri Kosina
Post by Clemens Kolbitsch
is there no way to tell the kernel, that a certain mapping must not be
removed, no matter what (except of course an explicit call to sys_unmap,
of course)?
I don't seem to get what is the issue here. Your mapping is not removed,
only the VMAs are merged together into one larger VMA if they have
neighbouring address ranges and compatible protection bits. See
vma_merge().
the thing is that they are not. the kernel chooses to REPLACE my mapping.
mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
here, the second call to mmap will shorten the first mapping to 0x1000 bytes
and create one big vma with size 0x5000 bytes.
is there a way to tell it that the second mmap MUST fail?
There's an LSM exit point for mmap, you could perhaps do something there.

What are you trying to achieve by forcing the second one to fail?
Clemens Kolbitsch
2007-08-30 22:10:08 UTC
Permalink
Post by V***@vt.edu
Post by Clemens Kolbitsch
Post by Jiri Kosina
Post by Clemens Kolbitsch
is there no way to tell the kernel, that a certain mapping must not
be removed, no matter what (except of course an explicit call to
sys_unmap, of course)?
I don't seem to get what is the issue here. Your mapping is not
removed, only the VMAs are merged together into one larger VMA if they
have neighbouring address ranges and compatible protection bits. See
vma_merge().
the thing is that they are not. the kernel chooses to REPLACE my mapping.
mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
here, the second call to mmap will shorten the first mapping to 0x1000
bytes and create one big vma with size 0x5000 bytes.
is there a way to tell it that the second mmap MUST fail?
There's an LSM exit point for mmap, you could perhaps do something there.
What are you trying to achieve by forcing the second one to fail?
puh... that is a good question :-)

I'm writing my master's thesis on a new model of memory protection and need to
have every memory mapping in userspace duplicated. I also have kind of a
second PGD/PTD that allows finding this mirrored mapping.

However, as the number of original mappings grows, I suddenly have the problem
that the kernel tries to allocate a new mapping and picks the address of a
mirrored memory page, which it shouldn't.

Honestly, I don't understand why it does so, but it simply does... so
basically I want these second mappings to stay in memory as long as the
original page.

What do you mean exactly with
Post by V***@vt.edu
There's an LSM exit point for mmap, you could perhaps do something there.
??
Post by V***@vt.edu
Which is exactly in compliance with what POSIX says about MAP_FIXED mmaps
- see http://opengroup.org/onlinepubs/007908799/xsh/mmap.html
I know... I'm not even saying that this is a bug... I just wonder if there is
some way to avoid this problem :-)

Thanks for your help - i really appreciate that!!
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Helge Hafting
2007-09-04 10:30:07 UTC
Permalink
Post by Clemens Kolbitsch
Post by V***@vt.edu
Post by Clemens Kolbitsch
Post by Jiri Kosina
Post by Clemens Kolbitsch
is there no way to tell the kernel, that a certain mapping must not
be removed, no matter what (except of course an explicit call to
sys_unmap, of course)?
I don't seem to get what is the issue here. Your mapping is not
removed, only the VMAs are merged together into one larger VMA if they
have neighbouring address ranges and compatible protection bits. See
vma_merge().
the thing is that they are not. the kernel chooses to REPLACE my mapping.
mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
here, the second call to mmap will shorten the first mapping to 0x1000
bytes and create one big vma with size 0x5000 bytes.
is there a way to tell it that the second mmap MUST fail?
There's an LSM exit point for mmap, you could perhaps do something there.
What are you trying to achieve by forcing the second one to fail?
puh... that is a good question :-)
I'm writing my master's thesis on a new model of memory protection and need to
have every memory mapping in userspace duplicated. I also have kind of a
second PGD/PTD that allows finding this mirrored mapping.
However, as the number of original mappings grows, I suddenly have the problem
that the kernel tries to allocate a new mapping and picks the address of a
mirrored memory page, which it shouldn't.
Honestly, I don't understand why it does so,
The "why" is easy: Having many mappings is expensive,
so merging them (when this cause no problems) is
a smart thing to do. So that is what linux does.
It means fewer mappings to keep track of. Less resources in
use means that linux moves faster.

If you are doing research, consider these methods:
1. Change vma_merge() so it always fail to merge mappings

or

2. Set up your "mappings duplicated in userspace" so
they too merge in the same way.

Helge Hafting
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Clemens Kolbitsch
2007-09-04 10:40:07 UTC
Permalink
Post by Helge Hafting
1. Change vma_merge() so it always fail to merge mappings
or
2. Set up your "mappings duplicated in userspace" so
they too merge in the same way.
Helge Hafting
Hi!
Thanks for your answer, however you (too) misunderstood: Merging of the vma's
is not the problem and I understand why the kernel does so.

The real problem is that it deletes mappings it should not be deleting... in
my case, I've found out that a possible reason for the deletions is that the
stack tries to grow or other large areas of memory are allocated and my
mapping is taking up the memory that the kernel wants to reserve.

However, this has been dealt with in some other reply - I decided to simply
relocate my mappings to another memory area as soon as the kernel tries to
delete them.

This is not really all that efficient, but it does not matter right now in my
case.

Thanks again for your answer - I appreciate it!




-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
linux-os (Dick Johnson)
2007-09-04 15:00:20 UTC
Permalink
Post by Clemens Kolbitsch
Post by V***@vt.edu
Post by Clemens Kolbitsch
Post by Jiri Kosina
Post by Clemens Kolbitsch
is there no way to tell the kernel, that a certain mapping must not
be removed, no matter what (except of course an explicit call to
sys_unmap, of course)?
I don't seem to get what is the issue here. Your mapping is not
removed, only the VMAs are merged together into one larger VMA if they
have neighbouring address ranges and compatible protection bits. See
vma_merge().
the thing is that they are not. the kernel chooses to REPLACE my mapping.
mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
here, the second call to mmap will shorten the first mapping to 0x1000
bytes and create one big vma with size 0x5000 bytes.
is there a way to tell it that the second mmap MUST fail?
There's an LSM exit point for mmap, you could perhaps do something there.
What are you trying to achieve by forcing the second one to fail?
puh... that is a good question :-)
I'm writing my master's thesis on a new model of memory protection and need to
have every memory mapping in userspace duplicated. I also have kind of a
second PGD/PTD that allows finding this mirrored mapping.
However, as the number of original mappings grows, I suddenly have the problem
that the kernel tries to allocate a new mapping and picks the address of a
mirrored memory page, which it shouldn't.
Honestly, I don't understand why it does so, but it simply does... so
basically I want these second mappings to stay in memory as long as the
original page.
What do you mean exactly with
Post by V***@vt.edu
There's an LSM exit point for mmap, you could perhaps do something there.
??
Post by V***@vt.edu
Which is exactly in compliance with what POSIX says about MAP_FIXED mmaps
- see http://opengroup.org/onlinepubs/007908799/xsh/mmap.html
I know... I'm not even saying that this is a bug... I just wonder if there is
some way to avoid this problem :-)
Thanks for your help - i really appreciate that!!
You might need to keep track of your memory mappings and
not remap something that will overlap. You can mmap
a single page in which you write the addresses. If you
map it shared, all your processes can access it and
call a common memory-mapping procedure (that you write)
which keeps track of its kernel allocations for you.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.22.1 on an i686 machine (5588.30 BogoMips).
My book : http://www.AbominableFirebug.com/
_


****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to ***@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Jiri Kosina
2007-08-30 22:00:13 UTC
Permalink
Post by Clemens Kolbitsch
the thing is that they are not. the kernel chooses to REPLACE my mapping.
mmap(0xaaaa0000, 0x3000, MAP_FIXED, ...);
mmap(0xaaaa1000, 0x4000, MAP_FIXED, ...);
here, the second call to mmap will shorten the first mapping to 0x1000 bytes
and create one big vma with size 0x5000 bytes.
Which is exactly in compliance with what POSIX says about MAP_FIXED mmaps
- see http://opengroup.org/onlinepubs/007908799/xsh/mmap.html
--
Jiri Kosina
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Continue reading on narkive:
Loading...