gmane.linux.kernel.mm

Re: [PATCH v3 00/28] kmem limitation for memcg

Michal Hocko — 2012-05-25T13:34:41

memcg-devel tree is only to make development easier. Everything that
applies on top of this tree should be applicable to both -mm and
linux-next.
So the patches should go via traditional Andrew's channel.

[PATCH v3 28/28] Documentation: add documentation for slab tracker for memcg

Glauber Costa — 2012-05-25T13:03:48

In a separate patch, to aid reviewers.

Signed-off-by: Glauber Costa <glommer< at >parallels.com>
CC: Christoph Lameter <cl< at >linux.com>
CC: Pekka Enberg <penberg< at >cs.helsinki.fi>
CC: Michal Hocko <mhocko< at >suse.cz>
CC: Kamezawa Hiroyuki <kamezawa.hiroyu< at >jp.fujitsu.com>
CC: Johannes Weiner <hannes< at >cmpxchg.org>
CC: Suleiman Souhlal <suleiman< at >google.com>
CC: Randy Dunlap <rdunlap< at >xenotime.net>
---
 Documentation/cgroups/memory.txt |   33 +++++++++++++++++++++++++++++++++
 1 files changed, 33 insertions(+), 0 deletions(-)

diff --git a/Documentation/cgroups/memory.txt b/Documentation/cgroups/memory.txt
index 4c95c00..9accaa1 100644
--- a/Documentation/cgroups/memory.txt
+++ b/Documentation/cgroups/memory.txt
< at >< at > -75,6 +75,12 < at >< at > Brief summary of control files.
  memory.kmem.tcp.limit_in_bytes  # set/show hard limit for tcp buf memory
  memory.kmem.tcp.usage_in_bytes  # show current tcp buf memory allocation
 
+ memory.kmem.limit_in_bytes # set/show hard limit for general kmem memory
+ memory.kmem.usage_in_bytes # show current general kmem memory allocation
+ memory.kmem.failcnt # show current number of kmem limit hits
+ memory.kmem.max_usage_in_bytes # show max kmem usage
+ memory.kmem.slabinfo # show cgroup-specific slab usage information
+
 1. History
 
 The memory controller has a long history. A request for comments for the memory
< at >< at > -271,6 +277,14 < at >< at > cgroup may or may not be accounted.
 Currently no soft limit is implemented for kernel memory. It is future work
 to trigger slab reclaim when those limits are reached.
 
+Kernel memory is not accounted until it is limited. Users that want to just
+track kernel memory usage can set the limit value to a big enough value so
+the limit is guaranteed to never hit. A kernel memory limit bigger than the
+current memory limit will have this effect as well.
+
+This guarantes that this extension is backwards compatible to any previous
+memory cgroup version.
+
 2.7.1 Current Kernel Memory resources accounted
 
 * sockets memory pressure: some sockets protocols have memory pressure
< at >< at > -279,6 +293,24 < at >< at > per cgroup, instead of globally.
 
 * tcp memory pressure: sockets memory pressure for the tcp protocol.
 
+* slab/kmalloc:
+
+When slab memory is tracked (memory.kmem.limit_in_bytes != -1ULL), both
+memory.kmem.usage_in_bytes and memory.usage_in_bytes are updated. When
+memory.kmem.limit_in_bytes is left alone, no tracking of slab caches takes
+place.
+
+Because a slab page is shared among many tasks, it is not possible to take
+any meaningful action upon task migration. Slabs created in a cgroup stay
+around until the cgroup is destructed. Information about the slabs used
+by the cgroup is displayed in the cgroup file memory.kmem.slabinfo. The format
+of this file is and should remain compatible with /proc/slabinfo.
+
+Upon cgroup destruction, slabs that holds no live references are destructed.
+Workers are fired to destroy the remaining caches as they objects are freed.
+
+Memory used by dead caches are shown in the proc file /proc/dead_slabinfo
+
 3. User Interface
 
 0. Configuration
< at >< at > -287,6 +319,7 < at >< at > a. Enable CONFIG_CGROUPS
 b. Enable CONFIG_RESOURCE_COUNTERS
 c. Enable CONFIG_CGROUP_MEM_RES_CTLR
 d. Enable CONFIG_CGROUP_MEM_RES_CTLR_SWAP (to use swap extension)
+d. Enable CONFIG_CGROUP_MEM_RES_CTLR_KMEM (to use experimental kmem extension)
 
 1. Prepare the cgroups (see cgroups.txt, Why are cgroups needed?)
 # mount -t tmpfs none /sys/fs/cgroup

[PATCH v3 27/28] memcg: propagate kmem limiting information to children

Glauber Costa — 2012-05-25T13:03:47

The current memcg slab cache management fails to present satisfatory hierarchical
behavior in the following scenario:

-> /cgroups/memory/A/B/C

* kmem limit set at A
* A and B empty taskwise
* bash in C does find /

Because kmem_accounted is a boolean that was not set for C, no accounting
would be done. This is, however, not what we expect.

The basic idea, is that when a cgroup is limited, we walk the tree
upwards (something Kame and I already thought about doing for other purposes),
and make sure that we store the information about the parent being limited in
kmem_accounted (that is turned into a bitmap: two booleans would not be space
efficient). The code for that is taken from sched/core.c. My reasons for not
putting it into a common place is to dodge the type issues that would arise
from a common implementation between memcg and the scheduler - but I think
that it should ultimately happen, so if you want me to do it now, let me
know.

We do the reverse operation when a formerly limited cgroup becomes unlimited.

Signed-off-by: Glauber Costa <glommer< at >parallels.com>
CC: Christoph Lameter <cl< at >linux.com>
CC: Pekka Enberg <penberg< at >cs.helsinki.fi>
CC: Michal Hocko <mhocko< at >suse.cz>
CC: Kamezawa Hiroyuki <kamezawa.hiroyu< at >jp.fujitsu.com>
CC: Johannes Weiner <hannes< at >cmpxchg.org>
CC: Suleiman Souhlal <suleiman< at >google.com>
---
 mm/memcontrol.c |  147 +++++++++++++++++++++++++++++++++++++++++++++++++------
 1 files changed, 131 insertions(+), 16 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 3e99c69..7572cb1 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
< at >< at > -259,6 +259,9 < at >< at > struct mem_cgroup {
  * the counter to account for kernel memory usage.
  */
 struct res_counter kmem;
+
+struct list_head children;
+struct list_head siblings;
 /*
  * Per cgroup active and inactive list, similar to the
  * per zone LRU lists.
< at >< at > -274,7 +277,11 < at >< at > struct mem_cgroup {
  * Should the accounting and control be hierarchical, per subtree?
  */
 bool use_hierarchy;
-bool kmem_accounted;
+/*
+ * bit0: accounted by this cgroup
+ * bit1: accounted by a parent.
+ */
+volatile unsigned long kmem_accounted;
 
 booloom_lock;
 atomic_tunder_oom;
< at >< at > -332,6 +339,9 < at >< at > struct mem_cgroup {
 #endif
 };
 
+#define KMEM_ACCOUNTED_THIS0
+#define KMEM_ACCOUNTED_PARENT1
+
 int memcg_css_id(struct mem_cgroup *memcg)
 {
 return css_id(&memcg->css);
< at >< at > -474,7 +484,7 < at >< at > void sock_release_memcg(struct sock *sk)
 
 static void disarm_static_keys(struct mem_cgroup *memcg)
 {
-if (memcg->kmem_accounted)
+if (test_bit(KMEM_ACCOUNTED_THIS, &memcg->kmem_accounted))
 static_key_slow_dec(&mem_cgroup_kmem_enabled_key);
 /*
  * This check can't live in kmem destruction function,
< at >< at > -4472,6 +4482,110 < at >< at > static ssize_t mem_cgroup_read(struct cgroup *cont, struct cftype *cft,
 len = scnprintf(str, sizeof(str), "%llu\n", (unsigned long long)val);
 return simple_read_from_buffer(buf, nbytes, ppos, str, len);
 }
+
+#ifdef CONFIG_CGROUP_MEM_RES_CTLR_KMEM
+typedef int (*memcg_visitor)(struct mem_cgroup*, void *);
+
+/*
+ * This is mostly "inspired" by the code in sched/core.c. I decided to copy it,
+ * instead of factoring it, because of all the typing issues we'd run into.
+ * In particular, grabbing the parent is very different for memcg, because we
+ * may or may not have hierarchy, while cpu cgroups always do. That would lead
+ * to either indirect calls - this is not a fast path for us, but can be for
+ * the scheduler - or a big and ugly macro.
+ *
+ * If we ever get rid of hierarchy, we could iterate over struct cgroup, and
+ * then it would cease to be a problem.
+ */
+int walk_tree_from(struct mem_cgroup *from,
+   memcg_visitor down, memcg_visitor up, void *data)
+{
+struct mem_cgroup *parent, *child;
+int ret;
+
+
+parent = from;
+down:
+ret = (*down)(parent, data);
+if (ret)
+goto out;
+
+list_for_each_entry_rcu(child, &parent->children, siblings) {
+parent = child;
+goto down;
+
+up:
+continue;
+}
+ret = (*up)(parent, data);
+if (ret || parent == from)
+goto out;
+
+child = parent;
+parent = parent_mem_cgroup(parent);
+if (parent)
+goto up;
+out:
+return ret;
+}
+
+static int memcg_nop(struct mem_cgroup *memcg, void *data)
+{
+return 0;
+}
+
+static int memcg_parent_account(struct mem_cgroup *memcg, void *data)
+{
+if (memcg == data)
+return 0;
+
+set_bit(KMEM_ACCOUNTED_PARENT, &memcg->kmem_accounted);
+return 0;
+}
+
+static int memcg_parent_no_account(struct mem_cgroup *memcg, void *data)
+{
+if (memcg == data)
+return 0;
+
+clear_bit(KMEM_ACCOUNTED_PARENT, &memcg->kmem_accounted);
+/*
+ * Stop propagation if we are accounted: our children should
+ * be parent-accounted
+ */
+return test_bit(KMEM_ACCOUNTED_THIS, &memcg->kmem_accounted);
+}
+
+static void mem_cgroup_update_kmem_limit(struct mem_cgroup *memcg, u64 val)
+{
+mutex_lock(&set_limit_mutex);
+if (!test_and_set_bit(KMEM_ACCOUNTED_THIS, &memcg->kmem_accounted) &&
+val != RESOURCE_MAX) {
+
+/*
+ * Once enabled, can't be disabled. We could in theory
+ * disable it if we haven't yet created any caches, or
+ * if we can shrink them all to death.
+ *
+ * But it is not worth the trouble
+ */
+static_key_slow_inc(&mem_cgroup_kmem_enabled_key);
+
+rcu_read_lock();
+walk_tree_from(memcg, memcg_parent_account, memcg_nop, memcg);
+rcu_read_unlock();
+} else if (test_and_clear_bit(KMEM_ACCOUNTED_THIS, &memcg->kmem_accounted)
+&& val == RESOURCE_MAX) {
+
+rcu_read_lock();
+walk_tree_from(memcg, memcg_parent_no_account,
+       memcg_nop, memcg);
+rcu_read_unlock();
+}
+
+mutex_unlock(&set_limit_mutex);
+}
+#endif
 /*
  * The user of this function is...
  * RES_LIMIT.
< at >< at > -4509,20 +4623,8 < at >< at > static int mem_cgroup_write(struct cgroup *cont, struct cftype *cft,
 ret = res_counter_set_limit(&memcg->kmem, val);
 if (ret)
 break;
-/*
- * Once enabled, can't be disabled. We could in theory
- * disable it if we haven't yet created any caches, or
- * if we can shrink them all to death.
- *
- * But it is not worth the trouble
- */
-mutex_lock(&set_limit_mutex);
-if (!memcg->kmem_accounted && val != RESOURCE_MAX
-    && !memcg->kmem_accounted) {
-static_key_slow_inc(&mem_cgroup_kmem_enabled_key);
-memcg->kmem_accounted = true;
-}
-mutex_unlock(&set_limit_mutex);
+mem_cgroup_update_kmem_limit(memcg, val);
+break;
 }
 #endif
 else
< at >< at > -5592,6 +5694,8 < at >< at > err_cleanup:
 
 }
 
+static DEFINE_MUTEX(memcg_list_mutex);
+
 static struct cgroup_subsys_state * __ref
 mem_cgroup_create(struct cgroup *cont)
 {
< at >< at > -5607,6 +5711,7 < at >< at > mem_cgroup_create(struct cgroup *cont)
 if (alloc_mem_cgroup_per_zone_info(memcg, node))
 goto free_out;
 
+INIT_LIST_HEAD(&memcg->children);
 /* root ? */
 if (cont->parent == NULL) {
 int cpu;
< at >< at > -5645,6 +5750,10 < at >< at > mem_cgroup_create(struct cgroup *cont)
  * mem_cgroup(see mem_cgroup_put).
  */
 mem_cgroup_get(parent);
+
+mutex_lock(&memcg_list_mutex);
+list_add_rcu(&memcg->siblings, &parent->children);
+mutex_unlock(&memcg_list_mutex);
 } else {
 res_counter_init(&memcg->res, NULL);
 res_counter_init(&memcg->memsw, NULL);
< at >< at > -5687,9 +5796,15 < at >< at > static int mem_cgroup_pre_destroy(struct cgroup *cont)
 static void mem_cgroup_destroy(struct cgroup *cont)
 {
 struct mem_cgroup *memcg = mem_cgroup_from_cont(cont);
+struct mem_cgroup *parent = parent_mem_cgroup(memcg);
 
 kmem_cgroup_destroy(memcg);
 
+mutex_lock(&memcg_list_mutex);
+if (parent)
+list_del_rcu(&memcg->siblings);
+mutex_unlock(&memcg_list_mutex);
+
 mem_cgroup_put(memcg);
 }

mm: fix faulty initialization in vmalloc_init()

KyongHo Cho — 2012-05-25T12:39:29

On Fri, May 25, 2012 at 7:12 AM, Andrew Morton <akpm< at >linux-foundation.org>
wrote:

Sorry for unkind commit message :)
Why this patch is needed is described by Olav
in the previous replies.


If it is not set, find_vm_area() with the early vm regions will always fail.

If the early vm regions must be neither found by find_vm_area()
nor removed by remove_vm_area(), va->vm must be NULL.

Please advise me what is right value for va->vm here :)

linux-samsung-soc" in

mm: fix faulty initialization in vmalloc_init()

KyongHo Cho — 2012-05-25T11:12:40

flags
gets
Thanks for description.

va->flags has different types of flags from tmp->flags.
If a region with VM_IOREMAP set is registered with vm_area_add_early(),
it will be removed by __purge_vmap_area_lazy().

 Cho KyongHo.

Re: [PATCH 0/2 v4] Flexible proportions

Fengguang Wu — 2012-05-25T09:29:36

Thank you both for making it work!  I've applied them to the writeback tree.

Thanks,
Fengguang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo< at >kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont< at >kvack.org"> email< at >kvack.org </a>

Re: [PATCH 0/2 v4] Flexible proportions

Peter Zijlstra — 2012-05-25T09:12:42

Thanks, all I could come up with is comment placement nits and I'll not
go there ;-)

Acked-by: Peter Zijlstra <a.p.zijlstra< at >chello.nl>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo< at >kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont< at >kvack.org"> email< at >kvack.org </a>

Re: [PATCH 2/2] vmevent: pass right attribute to vmevent_sample_attr()

Pekka Enberg — 2012-05-25T09:11:07

Right. I think Anton is reworking his patch which is why it's not merged 
to vmevent/core.

I applied your patch, thanks!

Pekka

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo< at >kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont< at >kvack.org"> email< at >kvack.org </a>

Re: [PATCH 2/2] vmevent: pass right attribute to vmevent_sample_attr()

Bartlomiej Zolnierkiewicz — 2012-05-25T08:15:06


Ah, sorry for that.  I generated this patch with Anton's "vmevent: Implement
special low-memory attribute" patch applied..  Here is a version against
vanilla vmevent-core:

From: Bartlomiej Zolnierkiewicz <b.zolnierkie< at >samsung.com>
Subject: [PATCH] vmevent: pass right attribute to vmevent_sample_attr()

Pass "config attribute" (&watch->config->attrs[i]) not "sample
attribute" (&watch->sample_attrs[i]) to vmevent_sample_attr() to
allow use of the original attribute value in vmevent_attr_sample_fn().

Cc: Anton Vorontsov <anton.vorontsov< at >linaro.org>
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie< at >samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park< at >samsung.com>
---
Without this patch vmevent_attr_lowmem_pages() always returns 0.

 mm/vmevent.c |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

Index: b/mm/vmevent.c
===================================================================
--- a/mm/vmevent.c2012-05-25 10:13:49.000000000 +0200
+++ b/mm/vmevent.c2012-05-25 10:13:56.231805967 +0200
< at >< at > -31,6 +31,7 < at >< at >
  */
 unsigned longnr_attrs;
 struct vmevent_attr*sample_attrs;
+struct vmevent_attr*config_attrs[VMEVENT_CONFIG_MAX_ATTRS];
 
 /* sampling */
 struct timer_listtimer;
< at >< at > -170,7 +171,8 < at >< at >
 for (i = 0; i < watch->nr_attrs; i++) {
 struct vmevent_attr *attr = &watch->sample_attrs[i];
 
-attr->value = vmevent_sample_attr(watch, attr);
+attr->value = vmevent_sample_attr(watch,
+  watch->config_attrs[i]);
 }
 
 atomic_set(&watch->pending, 1);
< at >< at > -313,6 +315,9 < at >< at >
 attrs[nr].type = attr->type;
 attrs[nr].value = 0;
 attrs[nr].state = 0;
+
+watch->config_attrs[nr] = attr;
+
 nr++;
 }
 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo< at >kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont< at >kvack.org"> email< at >kvack.org </a>

Re: [PATCH] vmevent: add arm support

Pekka Enberg — 2012-05-25T07:20:14


Applied, thanks!

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo< at >kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont< at >kvack.org"> email< at >kvack.org </a>

Re: [PATCH 2/2] vmevent: pass right attribute to vmevent_sample_attr()

Pekka Enberg — 2012-05-25T07:19:45


Looks good. I'm getting rejects for this. What tree did you use to 
generate the patch?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo< at >kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont< at >kvack.org"> email< at >kvack.org </a>

Re: [PATCH 1/2] vmevent: don't leak unitialized data to userspace

Pekka Enberg — 2012-05-25T07:17:16

Applied, thanks!

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo< at >kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont< at >kvack.org"> email< at >kvack.org </a>

RE: mm: fix faulty initialization in vmalloc_init()

Olav Haugan — 2012-05-25T00:24:01

 > -va->flags = tmp->flags | VM_VM_AREA;

I was actually debugging the same exact issue today. This transfer of flags
actually causes some of the static mapping virtual addresses to be
prematurely freed (before the mapping is removed) because VM_LAZY_FREE gets
"set" if tmp->flags has VM_IOREMAP set. This might cause subsequent
vmalloc/ioremap calls to fail because it might allocate one of the freed
virtual address ranges that aren't unmapped. 

--
Olav Haugan

Sent by an employee of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo< at >kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont< at >kvack.org"> email< at >kvack.org </a>

Re: mm: kernel BUG at mm/memory.c:1230

Sasha Levin — 2012-05-24T19:14:05

On Thu, May 24, 2012 at 9:07 PM, Andrew Morton
<akpm< at >linux-foundation.org> wrote:

Yup.


I'm not sure if that's indeed the issue or not, but note that this is
the first time I've managed to trigger that with the fuzzer, and it's
not that easy to reproduce. Which is a bit odd for code that was there
for 4 months...

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo< at >kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont< at >kvack.org"> email< at >kvack.org </a>

Re: mm: kernel BUG at mm/memory.c:1230

Andrew Morton — 2012-05-24T19:07:27

On Thu, 24 May 2012 20:27:34 +0200
Sasha Levin <levinsasha928< at >gmail.com> wrote:


That's

VM_BUG_ON(!rwsem_is_locked(&tlb->mm->mmap_sem));

in zap_pmd_range()?


The assertion was added in Jan 2011 by 14d1a55cd26f1860 ("thp: add
debug checks for mapcount related invariants").  AFAICT it's just wrong
on the exit path.  Unclear why it's triggering now...

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo< at >kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont< at >kvack.org"> email< at >kvack.org </a>

[PATCH 0/2 v4] Flexible proportions

Jan Kara — 2012-05-24T16:59:09

  Hello,

  here is the next iteration of my flexible proportions code. I've addressed
all Peter's comments.
Changes since v3:
  * changed fprop_fraction_foo() to avoid using percpu_counter_sum()
  * changed __fprop_inc_percpu_max() to avoid 64-bit division (now maximum
    allowed fraction is expressed max_frac/FPROP_FRAC_BASE)
  * avoid drifting of period timer
  * handle better cases where period timer fires long after intended time by
    aging by really passed number of periods
Changes since v2:
  * use timer instead of workqueue for triggering period switch
  * arm timer only if aging didn't zero out all fractions, re-arm timer when
    new event arrives again
  * set period length to 3s

  Some introduction for first time readers:

  The idea of this patch set is to provide code for computing event proportions
where aging period is not dependent on the number of events happening (so
that aging works well both with fast storage and slow USB sticks in the same
system).

  The basic idea is that we compute proportions as:
p_j = (\Sum_{i>=0} x_{i,j}/2^{i+1}) / (\Sum_{i>=0} x_i/2^{i+1})

  Where x_{i,j} is j's number of events in i-th last time period and x_i is
total number of events in i-th last time period.

  Note that when x_i's are all the same (as is the case with current
proportion code), this expression simplifies to the expression defining
current proportions which is:
p_j =  \Sum_{i>=0} x_{i,j}/2^{i+1} / t

  where t is the lenght of the aging period.

  In fact, if we are in the middle of the period, the proportion computed by
the current code is:
p_j = (x_0 + \Sum_{i>=1} x_{i,j}/2^{i+1}) / (t' + t)

  where t' is total number of events in the running period and t is the lenght
of the aging period. So there is event more similarity.

  Similarly as with current proportion code, it is simple to compute update
proportion after several periods have elapsed. For each proportion we store
the numerator of our fraction and the number of period when the proportion
was last updated. In global proportion structure we compute the denominator
of the fraction which is the same for all event types. So catch up with missed
periods boils down to shifting the numerator by the number of missed periods
and that's it. For more details, please see the code.

  I've also run a few tests (I've created a userspace wrapper to allow me to
run proportion code in userpace and arbitrarily generate events for it) to
compare the behavior of old and new code. You can see them at
http://beta.suse.com/private/jack/flex_proportions/ In all the tests new code
showed faster convergence to current event proportions (I tried to
realistically set period_shift for fixed proportions).  Also in the last test
we see that if period_shift is decreased, then current proportions become more
sensitive to short term fluctuations in event rate so just decreasing
period_shift isn't a good solution to slower convergence. If anyone has other
idea what to try, I can do that - it should be simple enough to implement in
my testing tool.

Honza

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo< at >kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont< at >kvack.org"> email< at >kvack.org </a>

[PATCH 1/2] lib: Proportions with flexible period

Jan Kara — 2012-05-24T16:59:10

Implement code computing proportions of events of different type (like code in
lib/proportions.c) but allowing periods to have different lengths. This allows
us to have aging periods of fixed wallclock time which gives better proportion
estimates given the hugely varying throughput of different devices - previous
measuring of aging period by number of events has the problem that a reasonable
period length for a system with low-end USB stick is not a reasonable period
length for a system with high-end storage array resulting either in too slow
proportion updates or too fluctuating proportion updates.

Signed-off-by: Jan Kara <jack< at >suse.cz>
---
 include/linux/flex_proportions.h |  100 ++++++++++++++
 lib/Makefile                     |    2 +-
 lib/flex_proportions.c           |  267 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 368 insertions(+), 1 deletions(-)
 create mode 100644 include/linux/flex_proportions.h
 create mode 100644 lib/flex_proportions.c

diff --git a/include/linux/flex_proportions.h b/include/linux/flex_proportions.h
new file mode 100644
index 0000000..9ac291c
--- /dev/null
+++ b/include/linux/flex_proportions.h
< at >< at > -0,0 +1,100 < at >< at >
+/*
+ * Floating proportions with flexible aging period
+ *
+ *  Copyright (C) 2011, SUSE, Jan Kara <jack< at >suse.cz>
+ */
+
+#ifndef _LINUX_FLEX_PROPORTIONS_H
+#define _LINUX_FLEX_PROPORTIONS_H
+
+#include <linux/percpu_counter.h>
+#include <linux/spinlock.h>
+#include <linux/seqlock.h>
+
+/*
+ * When maximum proportion of some event type is specified, this is the
+ * precision with which we allow limitting. Note that this creates an upper
+ * bound on the number of events per period like
+ *   ULLONG_MAX >> FPROP_FRAC_SHIFT.
+ */
+#define FPROP_FRAC_SHIFT 10
+#define FPROP_FRAC_BASE (1UL << FPROP_FRAC_SHIFT)
+
+/*
+ * ---- Global proportion definitions ----
+ */
+struct fprop_global {
+/* Number of events in the current period */
+struct percpu_counter events;
+/* Current period */
+unsigned int period;
+/* Synchronization with period transitions */
+seqcount_t sequence;
+};
+
+int fprop_global_init(struct fprop_global *p);
+void fprop_global_destroy(struct fprop_global *p);
+bool fprop_new_period(struct fprop_global *p, int periods);
+
+/*
+ *  ---- SINGLE ----
+ */
+struct fprop_local_single {
+/* the local events counter */
+unsigned long events;
+/* Period in which we last updated events */
+unsigned int period;
+raw_spinlock_t lock;/* Protect period and numerator */
+};
+
+#define INIT_FPROP_LOCAL_SINGLE(name)\
+{.lock = __RAW_SPIN_LOCK_UNLOCKED(name.lock),\
+}
+
+int fprop_local_init_single(struct fprop_local_single *pl);
+void __fprop_inc_single(struct fprop_global *p, struct fprop_local_single *pl);
+void fprop_fraction_single(struct fprop_global *p,
+struct fprop_local_single *pl, unsigned long *numerator,
+unsigned long *denominator);
+
+static inline
+void fprop_inc_single(struct fprop_global *p, struct fprop_local_single *pl)
+{
+unsigned long flags;
+
+local_irq_save(flags);
+__fprop_inc_single(p, pl);
+local_irq_restore(flags);
+}
+
+/*
+ * ---- PERCPU ----
+ */
+struct fprop_local_percpu {
+/* the local events counter */
+struct percpu_counter events;
+/* Period in which we last updated events */
+unsigned int period;
+raw_spinlock_t lock;/* Protect period and numerator */
+};
+
+int fprop_local_init_percpu(struct fprop_local_percpu *pl);
+void fprop_local_destroy_percpu(struct fprop_local_percpu *pl);
+void __fprop_inc_percpu(struct fprop_global *p, struct fprop_local_percpu *pl);
+void __fprop_inc_percpu_max(struct fprop_global *p, struct fprop_local_percpu *pl,
+    int max_frac);
+void fprop_fraction_percpu(struct fprop_global *p,
+struct fprop_local_percpu *pl, unsigned long *numerator,
+unsigned long *denominator);
+
+static inline
+void fprop_inc_percpu(struct fprop_global *p, struct fprop_local_percpu *pl)
+{
+unsigned long flags;
+
+local_irq_save(flags);
+__fprop_inc_percpu(p, pl);
+local_irq_restore(flags);
+}
+
+#endif
diff --git a/lib/Makefile b/lib/Makefile
index 74290c9..8dd81cf 100644
--- a/lib/Makefile
+++ b/lib/Makefile
< at >< at > -11,7 +11,7 < at >< at > lib-y := ctype.o string.o vsprintf.o cmdline.o \
  rbtree.o radix-tree.o dump_stack.o timerqueue.o\
  idr.o int_sqrt.o extable.o prio_tree.o \
  sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
- proportions.o prio_heap.o ratelimit.o show_mem.o \
+ proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
  is_single_threaded.o plist.o decompress.o
 
 lib-$(CONFIG_MMU) += ioremap.o
diff --git a/lib/flex_proportions.c b/lib/flex_proportions.c
new file mode 100644
index 0000000..530dbc2
--- /dev/null
+++ b/lib/flex_proportions.c
< at >< at > -0,0 +1,267 < at >< at >
+/*
+ *  Floating proportions with flexible aging period
+ *
+ *   Copyright (C) 2011, SUSE, Jan Kara <jack< at >suse.cz>
+ *
+ * The goal of this code is: Given different types of event, measure proportion
+ * of each type of event over time. The proportions are measured with
+ * exponentially decaying history to give smooth transitions. A formula
+ * expressing proportion of event of type 'j' is:
+ *
+ *   p_{j} = (\Sum_{i>=0} x_{i,j}/2^{i+1})/(\Sum_{i>=0} x_i/2^{i+1})
+ *
+ * Where x_{i,j} is j's number of events in i-th last time period and x_i is
+ * total number of events in i-th last time period.
+ *
+ * Note that p_{j}'s are normalised, i.e.
+ *
+ *   \Sum_{j} p_{j} = 1,
+ *
+ * This formula can be straightforwardly computed by maintaing denominator
+ * (let's call it 'd') and for each event type its numerator (let's call it
+ * 'n_j'). When an event of type 'j' happens, we simply need to do:
+ *   n_j++; d++;
+ *
+ * When a new period is declared, we could do:
+ *   d /= 2
+ *   for each j
+ *     n_j /= 2
+ *
+ * To avoid iteration over all event types, we instead shift numerator of event
+ * j lazily when someone asks for a proportion of event j or when event j
+ * occurs. This can bit trivially implemented by remembering last period in
+ * which something happened with proportion of type j.
+ */
+#include <linux/flex_proportions.h>
+
+int fprop_global_init(struct fprop_global *p)
+{
+int err;
+
+p->period = 0;
+/* Use 1 to avoid dealing with periods with 0 events... */
+err = percpu_counter_init(&p->events, 1);
+if (err)
+return err;
+seqcount_init(&p->sequence);
+return 0;
+}
+
+void fprop_global_destroy(struct fprop_global *p)
+{
+percpu_counter_destroy(&p->events);
+}
+
+/*
+ * Declare < at >periods new periods. It is upto the caller to make sure period
+ * transitions cannot happen in parallel.
+ *
+ * The function returns true if the proportions are still defined and false
+ * if aging zeroed out all events. This can be used to detect whether declaring
+ * further periods has any effect.
+ */
+bool fprop_new_period(struct fprop_global *p, int periods)
+{
+u64 events = percpu_counter_sum(&p->events);
+
+/*
+ * Don't do anything if there are no events.
+ */
+if (events <= 1)
+return false;
+write_seqcount_begin(&p->sequence);
+if (periods < 64)
+events -= events >> periods;
+/* Use addition to avoid losing events happening between sum and set */
+percpu_counter_add(&p->events, -events);
+p->period += periods;
+write_seqcount_end(&p->sequence);
+
+return true;
+}
+
+/*
+ * ---- SINGLE ----
+ */
+
+int fprop_local_init_single(struct fprop_local_single *pl)
+{
+pl->events = 0;
+pl->period = 0;
+raw_spin_lock_init(&pl->lock);
+return 0;
+}
+
+void fprop_local_destroy_single(struct fprop_local_single *pl)
+{
+}
+
+static void fprop_reflect_period_single(struct fprop_global *p,
+struct fprop_local_single *pl)
+{
+unsigned int period = p->period;
+unsigned long flags;
+
+/* Fast path - period didn't change */
+if (pl->period == period)
+return;
+raw_spin_lock_irqsave(&pl->lock, flags);
+/* Someone updated pl->period while we were spinning? */
+if (pl->period >= period) {
+raw_spin_unlock_irqrestore(&pl->lock, flags);
+return;
+}
+/* Aging zeroed our fraction? */
+if (period - pl->period < BITS_PER_LONG)
+pl->events >>= period - pl->period;
+else
+pl->events = 0;
+pl->period = period;
+raw_spin_unlock_irqrestore(&pl->lock, flags);
+}
+
+/* Event of type pl happened */
+void __fprop_inc_single(struct fprop_global *p, struct fprop_local_single *pl)
+{
+fprop_reflect_period_single(p, pl);
+pl->events++;
+percpu_counter_add(&p->events, 1);
+}
+
+/* Return fraction of events of type pl */
+void fprop_fraction_single(struct fprop_global *p,
+   struct fprop_local_single *pl,
+   unsigned long *numerator, unsigned long *denominator)
+{
+unsigned int seq;
+s64 num, den;
+
+do {
+seq = read_seqcount_begin(&p->sequence);
+fprop_reflect_period_single(p, pl);
+num = pl->events;
+den = percpu_counter_read_positive(&p->events);
+} while (read_seqcount_retry(&p->sequence, seq));
+
+/*
+ * Make fraction <= 1 and denominator > 0 even in presence of percpu
+ * counter errors
+ */
+if (den <= num) {
+if (num)
+den = num;
+else
+den = 1;
+}
+*denominator = den;
+*numerator = num;
+}
+
+/*
+ * ---- PERCPU ----
+ */
+#define PROP_BATCH (8*(1+ilog2(nr_cpu_ids)))
+
+int fprop_local_init_percpu(struct fprop_local_percpu *pl)
+{
+int err;
+
+err = percpu_counter_init(&pl->events, 0);
+if (err)
+return err;
+pl->period = 0;
+raw_spin_lock_init(&pl->lock);
+return 0;
+}
+
+void fprop_local_destroy_percpu(struct fprop_local_percpu *pl)
+{
+percpu_counter_destroy(&pl->events);
+}
+
+static void fprop_reflect_period_percpu(struct fprop_global *p,
+struct fprop_local_percpu *pl)
+{
+unsigned int period = p->period;
+unsigned long flags;
+
+/* Fast path - period didn't change */
+if (pl->period == period)
+return;
+raw_spin_lock_irqsave(&pl->lock, flags);
+/* Someone updated pl->period while we were spinning? */
+if (pl->period >= period) {
+raw_spin_unlock_irqrestore(&pl->lock, flags);
+return;
+}
+/* Aging zeroed our fraction? */
+if (period - pl->period < BITS_PER_LONG) {
+s64 val = percpu_counter_read(&pl->events);
+
+if (val < (nr_cpu_ids * PROP_BATCH))
+val = percpu_counter_sum(&pl->events);
+
+__percpu_counter_add(&pl->events,
+-val + (val >> (period-pl->period)), PROP_BATCH);
+} else
+percpu_counter_set(&pl->events, 0);
+pl->period = period;
+raw_spin_unlock_irqrestore(&pl->lock, flags);
+}
+
+/* Event of type pl happened */
+void __fprop_inc_percpu(struct fprop_global *p, struct fprop_local_percpu *pl)
+{
+fprop_reflect_period_percpu(p, pl);
+__percpu_counter_add(&pl->events, 1, PROP_BATCH);
+percpu_counter_add(&p->events, 1);
+}
+
+void fprop_fraction_percpu(struct fprop_global *p,
+   struct fprop_local_percpu *pl,
+   unsigned long *numerator, unsigned long *denominator)
+{
+unsigned int seq;
+s64 num, den;
+
+do {
+seq = read_seqcount_begin(&p->sequence);
+fprop_reflect_period_percpu(p, pl);
+num = percpu_counter_read_positive(&pl->events);
+den = percpu_counter_read_positive(&p->events);
+} while (read_seqcount_retry(&p->sequence, seq));
+
+/*
+ * Make fraction <= 1 and denominator > 0 even in presence of percpu
+ * counter errors
+ */
+if (den <= num) {
+if (num)
+den = num;
+else
+den = 1;
+}
+*denominator = den;
+*numerator = num;
+}
+
+/*
+ * Like __fprop_inc_percpu() except that event is counted only if the given
+ * type has fraction smaller than < at >max_frac/FPROP_FRAC_BASE
+ */
+void __fprop_inc_percpu_max(struct fprop_global *p,
+    struct fprop_local_percpu *pl, int max_frac)
+{
+if (unlikely(max_frac < FPROP_FRAC_BASE)) {
+unsigned long numerator, denominator;
+
+fprop_fraction_percpu(p, pl, &numerator, &denominator);
+if (numerator >
+    (((u64)denominator) * max_frac) >> FPROP_FRAC_SHIFT)
+return;
+} else
+fprop_reflect_period_percpu(p, pl);
+__percpu_counter_add(&pl->events, 1, PROP_BATCH);
+percpu_counter_add(&p->events, 1);
+}
+

[PATCH 2/2] block: Convert BDI proportion calculations to flexible proportions

Jan Kara — 2012-05-24T16:59:11

Convert calculations of proportion of writeback each bdi does to new flexible
proportion code. That allows us to use aging period of fixed wallclock time
which gives better proportion estimates given the hugely varying throughput of
different devices.

Signed-off-by: Jan Kara <jack< at >suse.cz>
---
 include/linux/backing-dev.h |    4 +-
 mm/backing-dev.c            |    6 +-
 mm/page-writeback.c         |  103 ++++++++++++++++++++++++++----------------
 3 files changed, 69 insertions(+), 44 deletions(-)

diff --git a/include/linux/backing-dev.h b/include/linux/backing-dev.h
index b1038bd..489de62 100644
--- a/include/linux/backing-dev.h
+++ b/include/linux/backing-dev.h
< at >< at > -10,7 +10,7 < at >< at >
 
 #include <linux/percpu_counter.h>
 #include <linux/log2.h>
-#include <linux/proportions.h>
+#include <linux/flex_proportions.h>
 #include <linux/kernel.h>
 #include <linux/fs.h>
 #include <linux/sched.h>
< at >< at > -89,7 +89,7 < at >< at > struct backing_dev_info {
 unsigned long dirty_ratelimit;
 unsigned long balanced_dirty_ratelimit;
 
-struct prop_local_percpu completions;
+struct fprop_local_percpu completions;
 int dirty_exceeded;
 
 unsigned int min_ratio;
diff --git a/mm/backing-dev.c b/mm/backing-dev.c
index dd8e2aa..3387aea 100644
--- a/mm/backing-dev.c
+++ b/mm/backing-dev.c
< at >< at > -677,7 +677,7 < at >< at > int bdi_init(struct backing_dev_info *bdi)
 
 bdi->min_ratio = 0;
 bdi->max_ratio = 100;
-bdi->max_prop_frac = PROP_FRAC_BASE;
+bdi->max_prop_frac = FPROP_FRAC_BASE;
 spin_lock_init(&bdi->wb_lock);
 INIT_LIST_HEAD(&bdi->bdi_list);
 INIT_LIST_HEAD(&bdi->work_list);
< at >< at > -700,7 +700,7 < at >< at > int bdi_init(struct backing_dev_info *bdi)
 bdi->write_bandwidth = INIT_BW;
 bdi->avg_write_bandwidth = INIT_BW;
 
-err = prop_local_init_percpu(&bdi->completions);
+err = fprop_local_init_percpu(&bdi->completions);
 
 if (err) {
 err:
< at >< at > -744,7 +744,7 < at >< at > void bdi_destroy(struct backing_dev_info *bdi)
 for (i = 0; i < NR_BDI_STAT_ITEMS; i++)
 percpu_counter_destroy(&bdi->bdi_stat[i]);
 
-prop_local_destroy_percpu(&bdi->completions);
+fprop_local_destroy_percpu(&bdi->completions);
 }
 EXPORT_SYMBOL(bdi_destroy);
 
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 26adea8..647daa3 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
< at >< at > -34,6 +34,7 < at >< at >
 #include <linux/syscalls.h>
 #include <linux/buffer_head.h> /* __set_page_dirty_buffers */
 #include <linux/pagevec.h>
+#include <linux/timer.h>
 #include <trace/events/writeback.h>
 
 /*
< at >< at > -135,7 +136,20 < at >< at > unsigned long global_dirty_limit;
  * measured in page writeback completions.
  *
  */
-static struct prop_descriptor vm_completions;
+static struct fprop_global writeout_completions;
+
+static void writeout_period(unsigned long t);
+/* Timer for aging of writeout_completions */
+static struct timer_list writeout_period_timer =
+TIMER_DEFERRED_INITIALIZER(writeout_period, 0, 0);
+static unsigned long writeout_period_time = 0;
+
+/*
+ * Length of period for aging writeout fractions of bdis. This is an
+ * arbitrarily chosen number. The longer the period, the slower fractions will
+ * reflect changes in current writeout rate.
+ */
+#define VM_COMPLETIONS_PERIOD_LEN (3*HZ)
 
 /*
  * Work out the current dirty-memory clamping and background writeout
< at >< at > -322,34 +336,6 < at >< at > bool zone_dirty_ok(struct zone *zone)
        zone_page_state(zone, NR_WRITEBACK) <= limit;
 }
 
-/*
- * couple the period to the dirty_ratio:
- *
- *   period/2 ~ roundup_pow_of_two(dirty limit)
- */
-static int calc_period_shift(void)
-{
-unsigned long dirty_total;
-
-if (vm_dirty_bytes)
-dirty_total = vm_dirty_bytes / PAGE_SIZE;
-else
-dirty_total = (vm_dirty_ratio * global_dirtyable_memory()) /
-100;
-return 2 + ilog2(dirty_total - 1);
-}
-
-/*
- * update the period when the dirty threshold changes.
- */
-static void update_completion_period(void)
-{
-int shift = calc_period_shift();
-prop_change_shift(&vm_completions, shift);
-
-writeback_set_ratelimit();
-}
-
 int dirty_background_ratio_handler(struct ctl_table *table, int write,
 void __user *buffer, size_t *lenp,
 loff_t *ppos)
< at >< at > -383,7 +369,7 < at >< at > int dirty_ratio_handler(struct ctl_table *table, int write,
 
 ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
 if (ret == 0 && write && vm_dirty_ratio != old_ratio) {
-update_completion_period();
+writeback_set_ratelimit();
 vm_dirty_bytes = 0;
 }
 return ret;
< at >< at > -398,12 +384,21 < at >< at > int dirty_bytes_handler(struct ctl_table *table, int write,
 
 ret = proc_doulongvec_minmax(table, write, buffer, lenp, ppos);
 if (ret == 0 && write && vm_dirty_bytes != old_bytes) {
-update_completion_period();
+writeback_set_ratelimit();
 vm_dirty_ratio = 0;
 }
 return ret;
 }
 
+static unsigned long wp_next_time(unsigned long cur_time)
+{
+cur_time += VM_COMPLETIONS_PERIOD_LEN;
+/* 0 has a special meaning... */
+if (!cur_time)
+return 1;
+return cur_time;
+}
+
 /*
  * Increment the BDI's writeout completion count and the global writeout
  * completion count. Called from test_clear_page_writeback().
< at >< at > -411,8 +406,19 < at >< at > int dirty_bytes_handler(struct ctl_table *table, int write,
 static inline void __bdi_writeout_inc(struct backing_dev_info *bdi)
 {
 __inc_bdi_stat(bdi, BDI_WRITTEN);
-__prop_inc_percpu_max(&vm_completions, &bdi->completions,
-      bdi->max_prop_frac);
+__fprop_inc_percpu_max(&writeout_completions, &bdi->completions,
+       bdi->max_prop_frac);
+/* First event after period switching was turned off? */
+if (!unlikely(writeout_period_time)) {
+/*
+ * We can race with other __bdi_writeout_inc calls here but
+ * it does not cause any harm since the resulting time when
+ * timer will fire and what is in writeout_period_time will be
+ * roughly the same.
+ */
+writeout_period_time = wp_next_time(jiffies);
+mod_timer(&writeout_period_timer, writeout_period_time);
+}
 }
 
 void bdi_writeout_inc(struct backing_dev_info *bdi)
< at >< at > -431,11 +437,33 < at >< at > EXPORT_SYMBOL_GPL(bdi_writeout_inc);
 static void bdi_writeout_fraction(struct backing_dev_info *bdi,
 long *numerator, long *denominator)
 {
-prop_fraction_percpu(&vm_completions, &bdi->completions,
+fprop_fraction_percpu(&writeout_completions, &bdi->completions,
 numerator, denominator);
 }
 
 /*
+ * On idle system, we can be called long after we scheduled because we use
+ * deferred timers so count with missed periods.
+ */
+static void writeout_period(unsigned long t)
+{
+int miss_periods = (jiffies - writeout_period_time) /
+ VM_COMPLETIONS_PERIOD_LEN;
+
+if (fprop_new_period(&writeout_completions, miss_periods + 1)) {
+writeout_period_time = wp_next_time(writeout_period_time +
+miss_periods * VM_COMPLETIONS_PERIOD_LEN);
+mod_timer(&writeout_period_timer, writeout_period_time);
+} else {
+/*
+ * Aging has zeroed all fractions. Stop wasting CPU on period
+ * updates.
+ */
+writeout_period_time = 0;
+}
+}
+
+/*
  * bdi_min_ratio keeps the sum of the minimum dirty shares of all
  * registered backing devices, which, for obvious reasons, can not
  * exceed 100%.
< at >< at > -475,7 +503,7 < at >< at > int bdi_set_max_ratio(struct backing_dev_info *bdi, unsigned max_ratio)
 ret = -EINVAL;
 } else {
 bdi->max_ratio = max_ratio;
-bdi->max_prop_frac = (PROP_FRAC_BASE * max_ratio) / 100;
+bdi->max_prop_frac = (FPROP_FRAC_BASE * max_ratio) / 100;
 }
 spin_unlock_bh(&bdi_lock);
 
< at >< at > -1605,13 +1633,10 < at >< at > static struct notifier_block __cpuinitdata ratelimit_nb = {
  */
 void __init page_writeback_init(void)
 {
-int shift;
-
 writeback_set_ratelimit();
 register_cpu_notifier(&ratelimit_nb);
 
-shift = calc_period_shift();
-prop_descriptor_init(&vm_completions, shift);
+fprop_global_init(&writeout_completions);
 }
 
 /**

the max size of block device on 32bit os,when using do_generic_file_read() proceed.

majianpeng — 2012-05-24T13:38:21

  Hi all:
I readed a raid5,which size 30T.OS is RHEL6 32bit.
    I reaed the raid5(as a whole,not parted) and found read address which not i wanted.
So I tested the newest kernel code,the problem is still.
I review the code, in function do_generic_file_read()

index = *ppos >> PAGE_CACHE_SHIFT;
index is u32.and *ppos is long long.
So when *ppos is larger than 0xFFFF FFFF *  PAGE_CACHE_SHIFT(16T Byte),then the index is error.

I wonder this .In 32bit os ,block devices size do not large then 16T,in other words, if block devices larger than 16T,must parted.

Thanks all.
 
--------------
majianpeng
2012-05-24

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo< at >kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont< at >kvack.org"> email< at >kvack.org </a>

Re: [PATCHv2 3/4] mm: vmalloc: add VM_DMA flag to indicate areas used by dma-mapping framework

Paul Mundt — 2012-05-24T12:28:54

There's very little about the code in question that is ARM-specific to
begin with. I plan to adopt similar changes on SH once the work has
settled one way or the other, so we'll probably use the VMA flag there,
too.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo< at >kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont< at >kvack.org"> email< at >kvack.org </a>

RE: [PATCHv2 3/4] mm: vmalloc: add VM_DMA flag to indicate areas used by dma-mapping framework

Marek Szyprowski — 2012-05-24T12:26:12

Hi Minchan,

On Tuesday, May 22, 2012 9:08 AM Minchan Kim wrote:


Right now yes, it will be used only on ARM architecture, but maybe other architecture will
start using it once it is available.


I've been told to avoid such #ifdef construction if there is no really good reason for it.
The only justification was significant impact on the performance, otherwise it would be 
just a good example of typical over-engineering.


Best regards