gmane.linux.debian.devel.general

Re: Moving /tmp to tmpfs is fine

Serge — 2012-05-25T12:25:58


Easy. Let's leave /tmp on a real disk and both world will "just work".


Or fix the program A, right? And here we go... By default the program
Debian is too memory-hungry (with large tmpfs) or breaking apps (with
small tmpfs). Let's fix that? ;)


Every system becomes "Low memory" with these defaults. Assuming you've
set your tmpfs size to 20% you need 2.5GB memory just to "temporarily"
unpack kernel sources and check for some files.

Or it's supposed to be unpacked not in /tmp? And other programs should
not be using /tmp to write large files? Then what *real* programs should
use /tmp now? Is it useless for popular programs?


So instead of fixing the defaults you suggest everybody to drop the
programs they use (mc, firefox, mysql)? ;)

Re: Moving /tmp to tmpfs is fine

Thorsten Glaser — 2012-05-25T11:58:14

Chow Loong Jin dixit:


Hm. mc does put things into /tmp as does debdiff, but the latter
at least honours TMPDIR (and --no-unpack-tarballs).

But in the very most cases, I *do* want them to be extracted in
/tmp as they “usually” fit. So I’d rather have a heuristic put
into the file manager whether to set TMPDIR before calling the
extraction utility (or which tmpdir to use if it designates the
extraction place by itself). mc maintainers, are you listening?

bye,
//mirabilos

Re: Moving /tmp to tmpfs makes it useless

Thorsten Glaser — 2012-05-25T11:44:15

Jon Dowland dixit:


No. An application might not know it’s writing to tmpfs (for
example, if it wasn’t even written for an operating system
with tmpfs in the first place). And it might want to use sync
writes. The user of such application might want to tell it to
write into a mounted tmpfs, for speed, if they don’t care about
data loss – and since sync is nop on tmpfs, this is A Good Thing.

bye,
//mirabilos

Re: Moving /tmp to tmpfs makes it useless

Roger Leigh — 2012-05-25T11:44:03

I haven't got anything particularly new to add to the discussion here.
But I would like to refer you to the previous discussion on the
topic.  I am well aware that the default does not satisfy all use
cases, but that's simply not possible for this problem.  The best we
can do is have a good default.  That could be by improving the
tmpfs default sizes and behaviour (my preferred solution).  It could
be by defaulting to not using a tmpfs.  However, the majority of
software which finds the tmpfs too small has unreasonable expectations
of what can be expected to be available (by default).

http://lists.debian.org/debian-devel/2011/04/msg00554.html
http://lists.debian.org/debian-devel/2011/11/msg00281.html
http://lists.debian.org/debian-devel/2012/02/msg00473.html

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=630615
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=630615#50
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=630615#89
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=665406
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=674517

http://fedoraproject.org/w/index.php?title=Features/tmp-on-tmpfs

Also see sysvinit in experimental.


Is this because the tmpfs is too small, or because those applications
are broken?  Please see the discussion above first.


I did want to have this for wheezy (#633299).  But I lacked the time
and familiarity with the d-i code, and the d-i developers also have
higher priorities.  I'm sure support for tmpfs in the partitioner would
be welcome if you want to add this functionality.


Regards,
Roger

Re: gitpkg with a quilt export hook

Bastien ROUCARIES — 2012-05-25T11:17:41

On Fri, May 25, 2012 at 1:32 AM, Brian May
<brian< at >microcomaustralia.com.au> wrote:

If you see the workflow, you could do this limiting the diff to debian
dir, and doing merging allow that debian sub branch (check with gitk
or qgit)

Branch are really really cheap

Moreover one branch one version allow me to easilly release to
stable/testing from experimental. If you use only one debian branch
you could not do this easilly.

Moreover you have a branch for every upstream revision used on debian
called upstream/version.

Bastien


.

Re: Moving /tmp to tmpfs makes it useful

Thorsten Glaser — 2012-05-25T11:04:54

Henrique de Moraes Holschuh dixit:


Not just that.

It’s faster because the files don’t even get written to disc if
they’re kept for, say, five minutes, and aren’t just short-lived.

If you’ve got a slow hard disc (say low-end spec or old system),
RAM’s still cheaper for tmpfs: no matter whether tmpfs or a real
on-disc filesystem, buffer cache would still get it, AND the writes
would be very slow (especially for short-lived files, _if_ the system
is otherwise busy and would not wait before writing them to disc),
and the system could concentrate on getting the files we want to
disc instead of the temporary ones.

And then there’s the CF card usecase. Or SSD, nowadays, I guess
(my IBM X40 just got two CF cards to replace the HDD). There, you’d
absolutely want to minimise disc writes. And just the chance that
it could not end up there (except as part of swap) is better then
the chance it could end up there (plus journal writes).


Right. /tmp always had size limitations, it used to be no bigger
than a few MiB, and people were ALWAYS tought to use /var/tmp for
files that are large, should not be removed across reboots or from
a cronjob pruning /tmp files older than 7 days, or a combination
of these. (Uglities like /usr/tmp notwithstanding, although I’ve
mostly seen that as compat symlink to /var/tmp later.)

I’m absolutely for /tmp as tmpfs by default. Even, no, especially
on low-end systems. Heck, be lucky it’s tmpfs and not Classical
BSDs’ mfs (basically a variable-size 4.2FFS ramdisc, not just
an object store in the buffer cache).

Every tool either supports setting TMPDIR=/var/tmp before running
it or is buggy. Go fix these instead, and then just run them with
that if you need them to process files you don’t want in tmpfs.

bye,
//mirabilos

Re: Moving /tmp to tmpfs is fine

Chow Loong Jin — 2012-05-25T10:40:08

That doesn't seem to happen with file-roller. Perhaps you need to file a bug
with your graphical archiver program.

Re: Moving /tmp to tmpfs makes it useless

Henrique de Moraes Holschuh — 2012-05-25T10:24:51

Switch to the deadline IO scheduler (and forget about any attempts at
fairness for desktop loads, but at least it never screws up as badly as
CFQ), and make proper use of cgroups. 

And disable memory overcommit, which actually *forces* you to have quite a
lot of both core and swap for things to even run... you have no idea how
much of a memory pig most crap nowadays is until you do.

Re: Moving /tmp to tmpfs makes it useless

Henrique de Moraes Holschuh — 2012-05-25T10:20:37

What IO scheduler are you using?  It must not be CFQ.  CFQ will happly
screw it up and stall both readers and writers when the number of dirty
pages gets too large (which will happen if you write a gigabyte file to
disk).

Either that, or you're cheating and your backend devices are simply "fast
enough" that it doesn't matter (fast RAID or fast SSD).

The real question is: what does it better under CFQ+IO contention?
Several threads doing filesystem IO, or the swapper?  Hint: it is not an
easy question to answer because it depends on the load, type of load,
and backend device.

Re: Moving /tmp to tmpfs is fine

Salvo Tomaselli — 2012-05-25T10:20:10

Double-click on a .tar causes it to be unpacked in /tmp/something.
I suppose a lot of not so skilled users do that instead of tar -xf

Can you link me one such device? Nothing pops in my mind. And how widespread 
they are?
Normal desktop/laptop configurations have at least 100 times more disk space 
than ram.
For servers.. it depends on the taks they are supposed to do, but usually 
servers have an administrator who is more or less aware of what he is doing.

True.
But experts can change the defaults more effectively than non experts.

The point is that the desktop user is often clueless and doesn't know it. He 
will just notice the problem and will not have a clue of how to solve it.

So I would suggest that defaulting to a more safe configuration would be 
better, and of course those who want tmpfs can always change the default. But 
those who don't even know about what /tmp is would not end up with something 
that doesn't work so well on their hardware.

Bye

Re: Moving /tmp to tmpfs makes it useless

Henrique de Moraes Holschuh — 2012-05-25T10:16:48

Hmm, no.  If the application Does It Right (emacs, vim...) they will
break hardlinks on open, and fsync properly on close/write by default.

That doesn't make them buggy, just because someone decided to tell them
to act on a file hosted in a tmpfs.

Re: Moving /tmp to tmpfs makes it useless

George Danchev — 2012-05-25T10:14:43


Agreed.


Unlikely. The last time I checked sync is a no-op on ramfs and tmpfs anyway 
(shmem.c). Also, tmpfs would not write out to swap on sync.

Likewise, as there is no point of syncing anything if a on-disk filesystem is 
mounted read-only, hence kernel does not even bother to (sync.c).

Re: Moving /tmp to tmpfs makes it useless

Henrique de Moraes Holschuh — 2012-05-25T10:14:20

Under heavy filesystem IO load, yes it is.  By several orders of magnitude.


Nice idea, but it would be worthless.

In fact it is the other way.  We have /var/tmp for the large file since
about forever, and important platforms that have /tmp in memory since the
early 2000's (Solaris)....

And that STILL wasn't enough for people to not screw it up and dump large
files in /tmp.

Re: Moving /tmp to tmpfs makes it useless

Salvo Tomaselli — 2012-05-25T09:54:57

In data Friday 25 May 2012 11:39:07, Josselin Mouette ha scritto:

I've never seen an almost full swap partition.
But the lack of memory has forced me to do several hardware resets 
nevertheless. And somehow the OOM killer was not even triggered, i kept ending 
up with a system where i had a working shell but could not run certain 
commands (such as kill), but i could see the output of "free" for example.

True, an insufficient swap is a cause of problems too. Not the only one tho.

Bye

Re: Moving /tmp to tmpfs is fine

Hendrik Sattler — 2012-05-25T09:30:04

Am 2012-05-25 11:19, schrieb Salvo Tomaselli:

Doing that on inferior hardware is just plain stupid. If you have 
plenty of
disk space, just unpack the tar archive.


And those with lots of RAM but not so much disk space (SD card or USB 
driver or
even with no hard drive at all)? There's no solution that works for 
everyone in
all situations. However, tmpfs at least works for many of them. If you 
KNOW
that this default does not fit your use-case, why don't you simply 
change the
configuration?

HS

Re: Moving /tmp to tmpfs makes it useless

Salvo Tomaselli — 2012-05-25T09:49:49

Yes because writing that on disk will only block the thread performing the 
write, not every thread that tries to allocate memory.

Re: Moving /tmp to tmpfs makes it useless

Josselin Mouette — 2012-05-25T09:41:26

Le vendredi 25 mai 2012 à 11:04 +0200, Salvo Tomaselli a écrit : 

Because paging out a couple Gigabytes is veeeeeery different from
writing a couple Gigabytes to disk, of course.

Re: Moving /tmp to tmpfs makes it useless

Josselin Mouette — 2012-05-25T09:39:07

Le vendredi 25 mai 2012 à 11:11 +0200, Salvo Tomaselli a écrit : 

Then increase your swap size.

When you run out of swap, the system fills memory with things that could
be swapped out, and has to reduce the buffer cache. 

Which then gives poor performance, regardless of the filesystem
your /tmp is mounted on.

Re: Moving /tmp to tmpfs makes it useless

Mehdi Dogguy — 2012-05-25T09:33:10

Increase your swap? If it is not possible, then your RAM won't help much
neither. If you really have many RAM and less disk, then this is very a 
specific case and we shouldn't bother making it "a general case".

Regards,

Re: Moving /tmp to tmpfs makes it useless

Charles Plessy — 2012-05-25T09:29:13

Hi all,

like many, I also felt like I had something to say in that thread.  But as
discussed in the “making debian-devel useful” thread(s), I will wait a bit to
see if others will not say it better before me.

Most importantly, it has been proposed many times that pepole limit themselves
to roughly one message per thread and per day when possible.  I think that it
would be a great way to make this thread constructive.

Conversly, people who have time to post many messages could perhaps spare some
to send a summary ?

It is not that a decision will be taken that weekend and that we need to rush
giving our opinions.  Also, I invite everybody to read Roger's proposition in
#630615 if they do not have time to read the whole bug discussion.

  http://bugs.debian.org/630615#89

Have a nice and relaxed week-end.

Re: Moving /tmp to tmpfs makes it useless

Andrei POPESCU — 2012-05-25T09:24:23


,----[ /usr/share/doc/initscripts/NEWS.Debian.gz ]
| sysvinit (2.88dsf-23) unstable; urgency=low
|
|   Changes to the configuration and defaults of tmpfs filesystems:
|
|   [...]
|
|   In order to improve the default size limits of tmpfs filesystems,
|   it is now possible to configure the size as a percentage of the
|   total virtual memory.  The default for /tmp and /run/shm is now
|   20%VM rather than 20% (RAM).  A tmpfs will only be mounted on
|   /tmp on systems with more than 64MiB RAM.
|
|  -- Roger Leigh <rleigh< at >debian.org>  Wed, 18 Apr 2012 23:30:37 +0100
`----

Kind regards,
Andrei