gmane.linux.lfs.beyond.devel

Re: ssd drives

Bruce Dubbs — 2012-05-25T03:01:04


Mine is Intel(R) Core(TM)2 Duo CPU E8400  < at > 3.00GHz
I have 3G of RAM.


It does say that in Section 2.4 for /mnt/lfs but I don't remember why.

I went back and searched the archives and see something about the glibc 
tests from 2003.


I agree.


Depends too on disk caching.


No, I think I'm about spent on this issue for now.

   -- Bruce

Re: ssd drives

Qrux — 2012-05-25T02:14:48

On May 24, 2012, at 6:19 PM, Bruce Dubbs wrote:


Interesting.  And unlike my experience...I barely observed more than
a 10 minute speedup over an entire automated build (using LFS, and then
building enough of BLFS to support Xen)--which took about 3 to 4 hours.

On my SSD test machine, my processor & memory weren't great.  i5-2400
with pretty tame memory.  Are yours much faster?  I wonder if my stuff
was system made the build CPU-bound...

Where does LFS stand now with respect to building with noatime?
I thought I read in 7.0 that that was bad news (I remember something
being atime-sensitive, and breaking; maybe it was the glibc or
coreutils test suite...)


I do agree that journaling is a whole other beast; i.e., if
we treat /usr and /bin as read-only (as a simplifying assumption)
then journaling should be entirely unnecessary, once they are created.

But...I think you may be missing the benefit of noatime.  The
purpose is to prevent write I/O when you just read files, by having
to update the inode on an open(2).  In fact, /usr should see the most
gain from noatime, because header files probably dumped from the
cache more often than, say, bash or gcc pages.

On a relatively complex operation like CMMI, which involves reading
hundreds (if not thousands) of files (package files, system includes,
system libraries, etc), I would expect noatime to have a pretty
significant difference.

Based on what you observed, I would bet that your ext3 partition
was mounted with relatime as the kernel default.  If that were the
case, you'd already be benefitting from read-side noatime.  And,
even though relatime doesn't exclude write-side atime, my guess
is that very few files are opened for writing more than once
(i.e., on creation), where recording atime would have practically
no impact.

You can test this by mounting the ext3 partition explicitly with
atime, then noatime, then with relatime (if you continue to be
interested in this).  :)

Q

Re: ssd drives

Bruce Dubbs — 2012-05-25T01:19:43


Not exactly.  GRUB is on sda and so is /boot.  GRUB reads the linux 
kernel from /boot and then the kernel initializes and sets up sdc1 as 
the root fs.


I'm not sure what you mean by login here.  The process is to power on 
(or reboot), have the BIOS do some initialization, and then run grub. 
The BIOS initialization (where we have no control) takes 2-3 seconds and 
then the GUUB prompt shows almost immediately after that.  From the GRUB 
selection to the linux login prompt takes a lot more than one second. 
Just running '/sbin/udevadm trigger --action=add --type=subsystems' 
takes considerably more than one second.


That's an interesting discussion.


I created a new ext4 partition and mounted it noatime.  My rsync build 
time was 20.3 seconds -- basically the same as ext3 with default mount 
options.

For things like /bin and /usr, I wouldn't be particularly concerned 
about atime and journaling since there is little or no writing (except 
when doing the install part of a build).

   -- Bruce

Re: ssd drives

Qrux — 2012-05-24T23:26:08

On May 24, 2012, at 3:38 PM, Bruce Dubbs wrote:


Are you booting from sdc (i.e., BIOS is booting from sdc)?

IDK what the rest of your setup is, but IIRC, I can get to login
(after BIOS POST, etc) in about a second.


Interesting data, but it might be a bit misleading to attribute all
the speedups to the SSD:

1) /tmp is ext3
2) /tmp is mounted with kernel defaults (relatime?)
3) / is mounted with data=writeback

1. Ext3 is probably far worse than ext4 for CMMI tasks.  Ext4 is
faster on file creation, and if it's used without barriers is going
to be much faster on writes.  Obviously the tradeoff is there is
filesystem safety.

2. Mounted with defaults, I'd guess that ext3 is probably using
relatime.  This is better than with atime enabled, but probably far
worse than noatime.  IIRC, relatime still writes atime, but just less
frequently.  It's still not as good as noatime.  Here's a casual
write-up (with references to the epic war on LKML):

http://tinyurl.com/2exlv7u

3. That same article quotes Torvalds, IIRC, in a heated LKML on
the noatime issue:

"But yeah, 'noatime,data=writeback' will quite likely be
*quite* noticeable (with different effects for different
loads), but almost nobody actually runs that way."

I think the context was about filesystem safety.  On a dev machine,
ext4 without barriers and data=writeback obviously makes sense.  But,
it's not something I'd suggest for anyone who is concerned about the
data on the drive.

Overall, I think the results would be more robust if you could run
the same test with ext3 with the same mount options, and see what
those results are.

Q

Re: ssd drives

Bruce Dubbs — 2012-05-24T22:38:24

I just want to share some things I've found out about ssd drives.

I installed the ssd drive as /dev/sdc and created a gpt partition table 
with parted.  The fist partition, /dev/sdc1, is about 11G.

I created a ext4 filesystem and mounted as /mnt/ssd.

Next I created the root directories: bin, boot, dev, etc, home,
lib, media, mnt, opt, proc, root, sbin, srv, sys, tmp, usr, var.

Then a symbolic link: lib64 -> lib

[/mnt/ssd] $ cp -a /etc/* etc
I did the same for bin, sbin, lib, usr, root

I updated /boot/grub/grub.cfg and found I needed some parameters on the 
boot line:

linux   /vmlinuz-3.3.6-lfs-20120515 root=/dev/sdc1 rootfstype=ext4 ro 
raid=noautodetect rootflags=data=writeback

The new items were rootfstype=ext4 and rootflags=data=writeback.

I did not initially create /run because I thought it was created in the 
first bootscript, but found that it couldn't be created because the 
filesystem is mounted as read only.  I'll need to remove that line in 
the boot script because it doesn't do anything.  (mkdir -p /run)

I also had problems initially because I forgot to create 
/dev/{console,null}.

I created separate partitions for /tmp and /var and copied the current 
/var to the new partition.

The fstab now looks like:

/dev/sdc1      /            ext4   noatime,discard,data=writeback  1  1
/dev/sda3      swap         swap   pri=1                0     0
/dev/sda1      /boot        ext3   defaults             1     2
/dev/sda9      /usr/src     ext3   defaults             1     2
/dev/sda11     /home        ext3   defaults             1     2
/dev/sdb5      /var         ext3   defaults             1     2
/dev/sdb6      /tmp         ext3   defaults             1     2

The kernel messages stopped at 3.998941 seconds and the bootscripts 
started.  The start time for all scripts was 6 seconds starting at 
16:45:15 for mountkernfs and finishing at 16:45:20 for sshd.

Previously the kernel was about about the same with the bootscripts 
running about 10 seconds.

I don't think putting /boot on the ssd would make much of a difference. 
  From the time I hit enter in grub to the time the kernel messages 
start is a second or less.

I then did a benchmark on rsync without the make check.  When building 
on /tmp, the CMMI time was 20 seconds.  When I changed the build to /mnt 
(on the ssd) the build time was 7.9 seconds.

Not bad.

   -- Bruce

Re: ssd drives

Bruce Dubbs — 2012-05-24T15:38:17

Of course there will be some variations, even on the same machine, due 
to other activities, caching, etc.  I'd say timings should, in general, 
be repeatable withing 5%, but could reach 10%.  More than that would 
mean something is happening like getting into swap.

   -- Bruce

Re: icedtea notes

Bruce Dubbs — 2012-05-24T15:33:31



What tools wolud that be?  I do not have gdb installed.  I did have all 
the 1.9.7 listed dependencies:  alsa-lib-1.0.25, Cups-1.5.0, 
gtk+-2.24.10, Xorg Libraries, xulrunner, apache-ant-1.8.3, UnZip-6.0, 
which-2.20, and Zip-3.0.  cpio is required too.

BTW, I think apache-ant is required, not optional.  At least I didn't 
see a way around it.  There is a circular dependency there.  My order was:

Tue May 22 11:21:21 2012 /usr/src/junit/junit4.10.zip
Tue May 22 11:24:56 2012 /usr/src/apache-ant/apache-ant-1.8.3-src.tar.bz2
Tue May 22 11:33:17 2012 /usr/src/cpio/cpio-2.11.tar.bz2
Tue May 22 11:38:15 2012 /usr/src/giflib/giflib-4.1.6.tar.bz2
Tue May 22 11:45:30 2012 /usr/src/cups/cups-1.5.0-source.tar.bz2
Tue May 22 20:09:25 2012 /usr/src/nspr/nspr-4.9.tar.gz
Tue May 22 20:17:32 2012 /usr/src/nss/nss-3.13.4.tar.gz
Wed May 23 02:13:58 2012 /usr/src/icedtea/icedtea-2.1.tar.gz

Looking at configure, there seem to be several dependencies we don't 
mention:  gcj, openssl, zlib, jpeg, gif, lcms, gtk, gio, fonconfig, but 
some of those are installed previously for xorg, etc,



I was using the xcfe wm and was running as non-root.  I did watch a lot 
of the tests, but I didn't see what I could have done to change the 
outcome of any particular test.


I think we would be better off to wait then.



A public results repo similar to what gcc has would be good.


I'm not sure I agree that a 3% error rate for the jdk portion of the 
tests is good.  I will note that over the years that gcc has gotten 
better and better about test results.


How is that done?


   -- Bruce

Re: ssd drives

Ken Moffat — 2012-05-24T13:39:27

 Umm, how *repeatable* they are.

ĸen

Re: icedtea notes

DJ Lucas — 2012-05-24T12:41:18

Hmm...JDI is the debug interface. I don't have an immediate explanation 
for that. Do you have traditional debugging tools available in the 
environment? They should not have an effect, but at the time I tested, I 
did not have them available.


This could be due to the window manager. I always sit and watch it 
through the tests. There were a couple where my login window was to 
blame when it was trying to iconify and restore windows. Also, root or 
unprivileged user? My results were obtained as an unprivileged user, 
using TWM on a fairly minimal system (just enough to meet required and 
recommended deps).

On a side note, 2.2 is scheduled for release next Wednesday (5/30). I 
don't anticipate any changes in the options or dependencies. 2.2 will 
basically be the same as the Oracle 7u4 with some of the security 
updates that are slotted for 7u5, along with the typical build fixes for 
newer system software and the closed parts of the software replaced by 
free software (easily arguable as _better_ replacements (pulse for the 
old sgi audio for example)). Basically, just disregard the upstream 
fixes patch and build as before. Existing 2.1 builds should work fine as 
a bootstrap compiler, however, we'll be pulling the downloads from git 
again (make download if you have wget installed, the all target will 
also get them for you in one step).

Unfortunately, until I hear back from the other maintainers on their 
test suite results, I really have nothing to go on. I know one of the 
core ITW developers had mentioned that he was working on a public repo 
for test results in private message, but as I understand it, he has 
about as much free time as I do lately. :-) Perhaps I should just 
install Fedora or Debian and build from their respective sources to make 
a meaningful comparison. In the grand scheme of things, the numbers are 
small in comparison to the number of tests, and not that different from 
what is in the book now which was deemed OK previously (by me using 
comparison with the other distros)...at least once the mystery is solved 
about the JDI tests. Maybe we should consider dropping the -samevm flag 
as one failure could conceivably cause a whole group of tests to fail if 
the running environment gets trashed. The downside to that is that a new 
VM is created for each test, which has a rather obvious effect on the 
wall clock.

Re: ssd drives

Ken Moffat — 2012-05-24T02:42:31

 Sure, I understand that.  But I imagine, perhaps wrongly, that
desktop environments will work better if they are on the fast disk.
So, using the SSD for / but with xorg and kde or qt in /opt on a
conventional drive would probably be slow to load.

 The other interesting feature about timings is how measurable they
are.  On my own machines there will be some cron jobs running from
time to time (disk checks with smartmontools - maybe not relevant
to the SSD but can be useful for conventional disks, log rotation,
backups, etc), plus whatever loadset is currently active [ think of
firefox as a memory hog ], plus any data in caches unless you drop
them.  For small packages, I expect to see variations of a few
seconds in build times.

 Anyway, enjoy it!

ĸen

Re: ssd drives

Bruce Dubbs — 2012-05-24T01:46:46

Thanks for the responses, Baho, Ken, and Qrux.  I did mount with 
noatime,discard,data=writeback and I also set hdparm -W1.  I think the 
important directories to have on a fast disk would be /bin, /lib, and 
/usr/{bin,lib}.  I'd think that any other files that may be on /etc or 
/usr/share would not show much difference.

At this point, I would not want to do extensive builds on the new drive 
as they do a lot of writing, but I probably will test a couple to see if 
it makes much difference.

As a quick initial test, I built rsync on the new drive and the build 
time, including tests, was not significantly different than from the 
standard disk (37.6 seconds vs 40.1 seconds).  The ssd build was 
actually slightly slower.

For Ken,  the reason I use /opt is primarily as an adjunct to 
/usr/{bin,lib}.  It can segregate a package and if I want to build a 
large package, say Qt, I can put it there and then upgrade it without 
worrying if there are version conflicts.  When I was working, I used Qt 
a lot and it was quite convenient, especially when transitioning from 
Qt3 to Qt4.  On my latest system, I have icedtea, xorg, and ant on /opt. 
  I use it as a crude package manager so that new packages can be 
installed and tested with the capability of easily backing up to older 
versions of a package if desired.

   -- Bruce

Re: ssd drives

Qrux — 2012-05-24T00:16:05

On May 23, 2012, at 4:00 PM, Bruce Dubbs wrote:


I would definitely avoid using it to build; that's probably what you
meant when you said to avoid /home.  It's faster (esp. for testing)
to use an in-memory filesystem if you find that your FS is a
bottleneck (which I haven't found).

Booting is awesome with SSDs.  So is having /bin and /usr on them.
Any setting where you spend oodles of time seeking is where you'll
see the most benefit with SSDs.  Yes, sustained reads are faster,
(though hdparm is pretty bad at measuring that; try using something
more purpose-built like iozone or bonnie++), but it's all about the
IOPS with SSDs.

I've used an SSD to store images of my host OS and the LFS downloads.
My host OS install blazes when I configure it to install from a
"hard drive partition" (which sits on the SSD) rather than from the CD.

I would also make sure to use noatime and make sure that I don't put
/var on it.  Maybe relatime is okay, but I'd rather have data on the
SSD, and avoid atime writes at all.  Obviously, don't put swap on it.

Q

Re: ssd drives

Ken Moffat — 2012-05-24T00:00:50

 If it was one of the cheap drives where the problems might be
slowness once it has to free up a lot of deleted space, and perhaps
a limit on how many times it can be updated before it fails, I'd be
inclined to try is as a r/o / but loaded from a real disk once the
system has all been built.  But it's NOT one of those, and I believe
Linus uses one as a normal disk.

 So, where will you benefit from the speed ?  In the old days /tmp
could have been useful, but now /tmp is RAM backed.  For me,
untarring and rm -rf to remove built packages is one of the big
delays.  So, *I* might be tempted to use it at /scratch (for me
that's where I put the "spare" space from my 500GB system disks - it
isn't backed up, unlike my /home, and it's where I build kernels, do
test builds of packages, and also where I will be doing media work
(transcoding etc).

 I recall that your partition usage is very different from mine, but
I think you need to look at the purpose of your partitions, not at
where you happen to mount them.  For me, /opt is for 'rubbish' that
I don't normally want to run (e.g. cmake, llvm) but sometimes am
forced to, but you use it for the current flavours of xorg and
desktop environments.

 If you want a fast system, all the programs you normally run, and
the configuration files they access, could be on the SSD.  That
would mean using partitions as / and /mnt/lfs.  You could even back
them up to the rotating disk "just in case".

 The problem with using it to measure package build times is that
the file i/o from compiling will be a lot faster so you might get
results that those of us on rotating drives can't achieve.  Not
necessarily a problem, and I'm fairly sure everyone will eventually
move to SSDs if they live up to expectations, but in the meantime
some of us might raise our eyebrows at your SBU times :-)

 Whatever you decide to do, I hope you enjoy it!  Must go, got to
see if my QT build has finished yet.

ĸen

Re: ssd drives

Baho Utot — 2012-05-23T23:52:24

I have been using an SSD and a reg HD for about 1 month now

here is how I setup the file system.

/etc/fstab
#    These part are on the SSD
/dev/sdb1 /boot ext4 defaults,noatime,discard 0 1
/dev/sdb2 / ext4 defaults,noatime,discard 0 1

#    These part are on the HD
/dev/mapper/lvm-home /home ext4 defaults,noatime 0 1
/dev/mapper/lvm-media /media ext4 defaults,noatime 0 1
/dev/mapper/lvm-swap swap swap defaults 0 0
/dev/mapper/lvm-var /var ext4 defaults,noatime 0 1

You should set discard on all the part that are on the SSD to enable TRIM.

ssd drives

Bruce Dubbs — 2012-05-23T23:00:14

I decided to explore using an ssd drive.  I purchased a 40G Intel SSD 
and so far it works well with SVN-20120514.

I created a GPT partition table with a 10G partition using parted and 
formatted as ext4.

The performance seems to be good.  htparm gives me about 230 GB/s which 
is more than twice as fast as my usual drives that give about 105 GB/s.

My question is how to best use the new drive in BLFS.  I thought of /opt 
and /usr.  I don't think /home would be very good and of course I could 
try to mount it as /mnt/lfs and use it as /.

What would you try first?

   -- Bruce

Re: icedtea notes

Bruce Dubbs — 2012-05-23T17:02:10


OK, here are some results:

Sources directory     195M
Build directory:      6.8G  (at one time grew to at least 7.2G)
/opt/icedtea-bin      423M
/opt/OpenJDK-1.7.0.3  422M
rhino1_7R3             17M
/usr/share/java       2.5M

Build time (minutes) 74:35  ( 44 SBU)
Test time (minutes) 216:59  (129 SBU)

Internally in the log I found:

icedtea notes

Bruce Dubbs — 2012-05-23T04:26:52

I've been working on icedtea today.  The notes DJ used seem to build 
fine, but there are a lot of errors in the tests.  I'm going through the 
build a second time.  The first time I tested via ssh and a lot of the 
tests were just timing out so I aborted and started over.

The second time I was in an xfce session and set:

xset s off
xset dpms 0 0 0

to keep the screen blanker from kicking in.

What I've noticed is that the reason the tests take so long is that many 
time out.  If a test is going to pass, it seems to do it fairly quickly, 
but there are a lot of failures where the system is just sleeping.  The 
load factors run down to very small values.

I'm still going through the tests for a second time and these are the 
failures so far:

FAILED: com/sun/awt/SecurityWarning/GetSizeShouldNotReturnZero.java
FAILED: com/sun/awt/Translucency/WindowOpacity.java
FAILED: com/sun/jdi/connect/spi/DebugUsingCustomConnector.java
FAILED: com/sun/jdi/redefine/RedefineTest.java
FAILED: com/sun/jdi/redefineMethod/RedefineTest.java
FAILED: com/sun/jdi/sde/FilterMangleTest.java
FAILED: com/sun/jdi/sde/MangleStepTest.java
FAILED: com/sun/jdi/sde/MangleTest.java
FAILED: com/sun/jdi/sde/SourceDebugExtensionTest.java
FAILED: com/sun/jdi/sde/TemperatureTableTest.java
FAILED: com/sun/jdi/AccessSpecifierTest.java
FAILED: com/sun/jdi/AfterThreadDeathTest.java

Right now the load factor is down to 0.26.  A lot of the failures seem 
to have something to do with threads/events.

The second time through the build/test not going through ssh seems to 
have a lot fewer errors but I've probably still got a couple of hours to 
go.  I'll give a final result tomorrow.

------------

The failures do have a log,  For instance the FilterMangleTest.jtr is 
the 'Java Test Report'.  It tells me the following, but I don't know how 
to interpret it.

   -- Bruce

Execution failed: `main' threw exception: java.lang.NullPointerException
...
RROR: transport error 202: connect failed: Connection timed out
ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510)
JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports 
initialized [../../../src/share/back/debugInit.c:741]
FATAL ERROR in native method: JDWP No transports initialized, 
jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197)
...

  Target VM failed to initialize.
java.lang.NullPointerException
    at VMConnection.open(VMConnection.java:196)
    at TestScaffold.connect(TestScaffold.java:632)
    at TestScaffold.startUp(TestScaffold.java:363)
    at TestScaffold.startTo(TestScaffold.java:373)
    at TestScaffold.startToMain(TestScaffold.java:368)
    at FilterMangleTest.runTests(FilterMangleTest.java:160)
    at TestScaffold.startTests(TestScaffold.java:429)
    at FilterMangleTest.main(FilterMangleTest.java:96)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at 
com.sun.javatest.regtest.MainWrapper$MainThread.run(MainWrapper.java:94)
    at java.lang.Thread.run(Thread.java:722)

nss

Bruce Dubbs — 2012-05-23T01:24:38

Looking at nss, the intro says:

* Estimated disk space required: 70 MB (more than double this to run the 
test suite)

* Estimated build time: 1.0 SBU (at least an additional 3.5 SBU to run 
the test suite)

But the instructions say:

   "This package does not come with a test suite."

I looked at the Makefile and could not any test or check.  Is the intro 
data about a test suite just an oversight from prior versions?

The size and md5sum were correct and the build size and time were pretty 
close to what I had.

   -- Bruce

Re: colord check

Bruce Dubbs — 2012-05-22T15:43:35


Thanks for the feedback. I suspect we should use 'make -k check' in the 
book and say that there may be isolated failures.

   -- Bruce

Re: colord check

Wayne Blaszczyk — 2012-05-22T09:18:28

Not that particular error but I do get the following error on one
workstation:
/colord/device-duplicate: **
libcolord:ERROR:cd-self-test.c:1965:colord_device_duplicate_func:
assertion failed (error == NULL):
GDBus.Error:org.freedesktop.ColorManager.Failed: failed to obtain
org.freedesktop.color-manager.create-device auth (cd_client_error, 0)
/bin/sh: line 5:  9957 Aborted                 ${dir}$tst
FAIL: cd-self-test

whilst on my main workstation I get a bit further:
/colord/client-random: **
libcolord:ERROR:cd-self-test.c:393:colord_client_random_func: assertion
failed (cd_profile_get_created (profile) == 1261606846): (1261570846 ==
1261606846)
/bin/sh: line 5:  8179 Aborted                 ${dir}$tst
FAIL: cd-self-test
I normally don't pay attention to test failures. I do however have Vala
API generator set to yes and also libgusb which is not mentioned in the
book.
Wayne.

colord check

Bruce Dubbs — 2012-05-21T21:13:34

I'm getting a failure in 'make check' in colord:

libcolord:ERROR:cd-self-test.c:971:colord_client_func: assertion failed 
(error == NULL): Could not connect: No such file or directory 
(g-io-error-quark, 1)
/bin/sh: line 5: 18614 Aborted                 ${dir}$tst
FAIL: cd-self-test

Has anyone else seen this?  The only dependency I haven't built is sane.

                     colord 0.1.19
                   ===================

         prefix:                    /usr
         datadir:                   ${datarootdir}
         compiler:                  gcc
         cflags:                    -g -O2
         cppflags:
         gobject-introspection:     yes
         PolicyKit support:         yes
         Reverse engineering tools: no
         File descriptor fallback:  yes
         External volume searching: yes
         SANE support:              no
         GUDEV support:             yes
         LCMS2 DICT support:
         Vala API generator:        no
         Daemon user:               root


   -- Bruce