gmane.os.netbsd.ports.ofppc

Apply for loan < at > 2%...

Apna-Loan — 2013-04-09T19:40:38

Please Contact Us With This Email: apnapaisaloan.com12< at >hotmail.com

Apna Paisa Loan Company , ? Are you in any financial mess or do you 

need a loan to start up your own business? at 2% rate? ; Email 

:apnapaisaloan.com12< at >hotmail.com

(1) Full Names:
(2) State/Country:
(3)Amount needed as loan):
(4)Loan duration:
(5)Cell-Phone number:

Mr. Harsh Roongta

Re: pegasosII hung with new kernel - page daemon spinning

Matt Thomas — 2012-09-23T16:03:24

On Sep 23, 2012, at 1:39 AM, Lars Heidieker wrote:

PPC will try to use direct mapped memory for uareas so those won't
eat KMEM.  In fact, since PMAP_MAP_POOLPAGE is defined, a lot of
KMEM usage is reduced.

Re: pegasosII hung with new kernel - page daemon spinning

Lars Heidieker — 2012-09-23T08:39:55

ah I see we have 2 segments for the kernel, going to high here will most
likely limit the memory for uareas.
I would suggest give it a try with 200mb and see if the arenas free
memory drops below 10%.

Lars

Re: pegasosII hung with new kernel - page daemon spinning

Lars Heidieker — 2012-09-23T08:09:46

Yes, that's what I think. Having the limit at 1/3 of the kernel address
space size make sense.

I don't know for sure but the kernel address space and user address
space on powerpc are disjunct right? So we are not really limited at all
like on i386.

Lars

re: pegasosII hung with new kernel - page daemon spinning

matthew green — 2012-09-22T23:54:38


i noticed macppc already defines this to 256MB.  trying that now.


.mrg.

Re: pegasosII hung with new kernel - page daemon spinning

Lars Heidieker — 2012-09-22T22:58:01

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/23/2012 12:51 AM, matthew green wrote:

That's interessting what size does the arena have on the system with
512mb? Probably limited by NKMEMPAGES_MAX_DEFAULT, so it might need to
be larger on those.

Lars

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlBeQnkACgkQcxuYqjT7GRZeugCeI1y+2DoWyyVn6o79lZwNxc3h
cCgAoKS3CvCcHEVunuU4koBznWvi3iFg
=gdl4
-----END PGP SIGNATURE-----

pegasosII hung with new kernel - page daemon spinning

matthew green — 2012-09-22T22:26:43

hi folks.


testing out the ppc intr fix i ran 'cvs up -dPA' in my src tree and
a short while later, my system has soft-locked up.  it appears that
the page daemon is cpu spinning not giving up the cpu.  but there's
plenty of free pages and etc so it shouldn't be rurnning all the
time:

db> show uvmexp
Current UVM status:
  pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
, ncolors=1  123256 VM pages: 6215 active, 3035 inactive, 0 wired, 96063 free
  pages  2360 anon, 5034 file, 1856 exec
  freemin=256, free-target=341, wired-max=41085
  cpu0:
    faults=46153, traps=48555, intrs=7562381, ctxswitch=58644
    softint=1100268, syscalls=0
  fault counts:
    noram=0, noanon=0, pgwait=0, pgrele=0
    ok relocks(total)=488(488), anget(retrys)=13157(0), amapcopy=5069
    neighbor anon/obj pg=11116/81215, gets(lock/unlock)=21061/488
    cases: anon=8239, anoncow=4906, obj=18107, prcopy=2954, przero=7119
  daemon and swap counts:
    woke=1, revs=2, scans=0, obscans=0, anscans=0
    busy=0, freed=0, reactivate=

Re: Current port status

Frank Wille — 2012-04-18T14:41:37

On Tue, 17 Apr 2012 03:43:48 +1000
matthew green <mrg< at >eterna.com.au> wrote:


Yes, that what I thought too.



Neither am I. Should we forward the problem to tech-kern?

AFAICS vr(4) does its m_get() in vr_rxeof() since more than a decade.
So probably something in the kernel had changed now, which makes the
problem visible.

re: Current port status

matthew green — 2012-04-16T17:43:48


yeah, it was strange at first to me.  i think the problem is that
m_get() is being called at both IPL_SOFTNET (the first m_get() in
the stack trace, from softint) and IPL_VM (the second, from beyond
vr_intr().)  ie, vr(4) shouldn't be allocating from the intr.  but
i'm not really a network driver person...

the problem about completely released is that:

- before releasing the mutex, we check it is properly held

- this check, in lockdebug, runs under splhigh()

- when this code returns, and calls splx() the intr is
  services

- the very next line after the lockdebug check is the
  release of the underlying simple lock.

it's all ugly but i think correct, and a vr(4) bug as we have
been suspecting.


.mrg.

Re: Current port status

Frank Wille — 2012-04-16T11:25:13


This is something I don't understand. Obviously mutex_exit() had been called
in [1] for this pool, so why didn't it have been "completely released" when
entering again in [5]?

I don't know very much about the internals of kernel mutexes, but when
mutex_exit() would really call splx() before having released everything
this sounds like a bug in mutex_exit(), which is hard to believe as it
works in most other environments.

Re: Current port status

Frank Wille — 2012-04-16T11:18:40


Yes, but why? It's hard to believe this is a MI problem in pools. So
probably vr(4) did something bad?



329950 is after a mutex_enter() in pool_get(). And 329acc after a
mutex_exit() in the same function.

Another example has:
last locked: 32ac08  unlocked*: 32ad80
(so the star is attached to unlocked - what does it mean?).

Here the mutex_enter() and mutex_exit() were called from
pool_cache_get_slow().

re: Current port status

matthew green — 2012-04-15T21:27:37

with some help from rmind< at > i've at least tracked down what is happening
if not why or what the fix is.  i got this lockdebug panic:

Mutex error: mutex_vector_enter: locking against myself

lock address : 0x00000000a01ea074 type     :               spin
initialized  : 0x00000000003c4bfc
shared holds :                  0 exclusive:                  0
shares wanted:                  0 exclusive:                  1
current cpu  :                  0 last held:                  0
current lwp  : 0x00000000a0007840 last held: 000000000000000000
last locked  : 0x00000000003c5128 unlocked*: 0x00000000003c52a4
owner field  : 000000000000000000 wait/spin:                0/1

panic: LOCKDEBUG
Stopped in pid 0.3 (system) at  netbsd:cpu_Debugger+0x10:       lwz     r0, 0x14(r1)
db> bt
0xa94a3940: at vpanic+0x21c
0xa94a3970: at panic+0x4c
0xa94a39b0: at lockdebug_abort1+0xdc
0xa94a39d0: at mutex_abort+0x50
0xa94a39e0: at mutex_enter+0x26c                          [5]
0xa94a3a20: at pool_get+0x7c

re: Current port status

matthew green — 2012-04-15T20:02:28


hmmmm.  this looks like the same "full mag" problem:

                if (__predict_true(pcg->pcg_avail > 0)) {
                        object = pcg->pcg_objects[--pcg->pcg_avail].pcgo_va;
...
                        KASSERT(object != NULL);


i haven't seen this one, but i haven't pushed much.  i'll try more.
can you find out where the 0x00329950 and 0x00329acc code points are?

thanks.


.mrg.

Re: Current port status

Frank Wille — 2012-04-15T11:33:09


Ok, now I can reproduce it! Sorry, I should have noticed that this output
can only occur with DEBUG and/or DIAGNOSTIC. :)

I don't think it has something to do with pools or NFS, but more likely with
the vr(4) driver. I could reproduce all kinds of kernel-assertion- and
lockdebug-panics with a simple FTP transfer over vr(4).

For example I could reproduce a similar diagnostic assertion as yours after
running into the infamous "vr0: device timeout" during an FTP file
transfer. Then I pressed CTRL-C to break, and this happened (manually
copied from screen):

ftp> get etc.tgz
vr0: device timeout
...
CTRL-C
receive aborted. Waiting for remote to finish abort.
panic: kernel diagnostic assertion "cur->pcg_avail == cur->pcg_size" failed:
file "/home/frank/netbsd/current/src/sys/kern/subr_pool.c", line 2573
...
db> bt
vpanic+0x21c
kern_assert+0x68
pool_cache_put_slow+0x30c
pool_cache_put_paddr+0x18c
m_ext_free+0xf0
soreceive+0xcc4
soo_read+0x28
dofileread+0x8c
syscall_plain+0x1f8
user SC trap #3 by 0xfde30b3c

Not

re: Current port status

matthew green — 2012-04-14T20:06:18


ah...nevermind about the build comment.  it's running the .56 kernel.

re: Current port status

matthew green — 2012-04-14T19:15:48


what kernel options do you use?  i am using these, plus more:

include "arch/ofppc/conf/GENERIC"
options         DEBUG
options         LOCKDEBUG
options         DIAGNOSTIC
options         KMEM_GUARD_DEPTH=0x1000

which is probably necessary for this.


.mrg.

re: Current port status

matthew green — 2012-04-14T19:08:28


originall cp(1) but now i just use "what /nk" (/nk is a symlink to
the right place), which crashes repeatably for me.


vr(4).


my 5.99.56 kernel seems pretty stable.  i haven't noticed any new vr(4)
issues there.  my 6.99.4 kernel seems stable when using NFS for /usr/src
and running a build, it's only these kernels (large files?)


oh, i'll release :-)

Re: Current port status

Frank Wille — 2012-04-14T12:24:05


What do you mean by "read"? Are you just copying the kernel from NFS with
cp(1), or is the kernel booted from NFS?



I couldn't reproduce that during the last days. Neither by running
standalone from NFS nor by copying a kernel or other large files from NFS.


Which network interface did you use? vr(4) or mvgbe(4)?

With the recent 6_beta I realized that vr(4) has become nearly unusable and
frequently gets stuck with "vr0: device timeout". This has been observed
since 5.99.59, but not before 5.99.42.

I have neither the time to look into all that nor am I experienced enough to
fix all the open issues, so I fear that there will be no ofppc release for
6.0.

re: Current port status

matthew green — 2012-04-08T03:14:42

i've had a new crash.  i saw it with a 5.99.65 kernel and a
very -current kernel, when trying to read a new kernel over NFS
from the build machine:

panic: kernel diagnostic assertion "cur->pcg_avail == cur->pcg_size" failed: file "/usr/src/sys/kern/subr_pool.c", line 2573
Stopped in pid 0.45 (system) at netbsd:cpu_Debugger+0x10:       lwz     r0, 0x14(r1)
db> bt
0xaa45bc90: at vpanic+0x21c
0xaa45bcc0: at kern_assert+0x68
0xaa45bd00: at pool_cache_put_slow+0x30c
0xaa45bd30: at pool_cache_put_paddr+0x18c
0xaa45bd60: at m_ext_free+0xf0
0xaa45bd70: at m_freem+0xd4
0xaa45bda0: at nfs_readrpc+0x3ec
0xaa45be30: at nfs_doio+0x694
0xaa45bee0: at nfssvc_iod+0x1c4
0xaa45bf20: at cpu_lwp_bootstrap+0xc
saved LR(0x2) is invalid.

not much idea about it yet.  the assert means that because
we've taken the slow path, the cache should be full.  100%
repeatable for me.


.mrg.

Re: Current port status

Matt Thomas — 2012-02-05T01:25:21

On Feb 4, 2012, at 3:38 PM, Frank Wille wrote:



I did it on vr0.

Re: Current port status

Frank Wille — 2012-02-04T23:38:35

On Sun, 05 Feb 2012 10:03:47 +1100
matthew green <mrg< at >eterna.com.au> wrote:


I haven't seen this panic yet. What do you have to do to create it?
Just run cvs on mvgbe0?