http://permalink.gmane.org/gmane.comp.parsers.sparse hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://permalink.gmane.org/gmane.comp.parsers.sparse/2859 <pre> substitute_arguments() is because I am not allowed extra token TOKEN_M_EMPTY. it is unrelated to the upper cases... I'm not shure who is wrong here or weather maybe my example is not general enought, maybe you are right... Kind-of:-) http://gcc.gnu.org/ml/gcc/2012-03/msg00208.html It disapears in the gcc list noise however... I dump everything, however I use a perl script to reconstruct... -- To unsubscribe from this list: send the line "unsubscribe linux-sparse" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Konrad Eisele 2012-05-15T13:03:49 http://permalink.gmane.org/gmane.comp.parsers.sparse/2858 <pre>Hi, I am resending the patch with the reference to Linus' email added to Documentation/sparse.txt. Thanks for all your feedback and replies. Signed-off-by: Shakthi Kannan &lt;shakthimaan&lt; at &gt;gmail.com&gt; --- Documentation/sparse.txt | 45 +++++++++++++++++++++++++++++++++++++++++++++ sparse.1 | 14 +++++++++++++- 2 files changed, 58 insertions(+), 1 deletions(-) create mode 100644 Documentation/sparse.txt diff --git a/Documentation/sparse.txt b/Documentation/sparse.txt new file mode 100644 index 0000000..1ee3b39 --- /dev/null +++ b/Documentation/sparse.txt &lt; at &gt;&lt; at &gt; -0,0 +1,45 &lt; at &gt;&lt; at &gt; +Sparse +~~~~~~ + +__nocast vs __bitwise: + +__nocast warns about explicit or implicit casting to different types. + +HOWEVER, it doesn't consider two 32-bit integers to be different +types, so a __nocast 'int' type may be returned as a regular 'int' +type and then the __nocast is lost. + +So "__nocast" on integer types is usually not that powerful. It just +gets lost too easily. It's more useful for things like pointers. </pre> Shakthi Kannan 2012-05-15T10:06:36 http://permalink.gmane.org/gmane.comp.parsers.sparse/2857 <pre> OK, I mistaken that as apply request. Never mind some of the coding style comment then. At this point I think 1) is actually better for your requirement. I start out as hoping the expand_macro() can abstract away the internal implementation detail of macro expand and just give you the text before and after the macro expand. But with all this extra manipulations of the macro tokens, especially the substitute_argument() is clear indicate tightening into the implementation details. I would take 1) over substitute_arguments() if 1) don't need call back like substitute_arguments(). That is surprising to me. I previously have different assumptions. I assume you want to back trace all the way back to the source macro. I would just say main() depend on xdef, which depend one S1 and S2. S1 also depend on E1 and S2 depend on E2. If you bypass xdef and directly extract S1(sx). Why can't you do one step further bypass S1() as well, and say main depend on "struct sx { E1 int d1;}? This is even more complicated</pre> Christopher Li 2012-05-15T09:44:49 http://permalink.gmane.org/gmane.comp.parsers.sparse/2856 <pre>The last patch with mdep.c and test-mdep.c was also nothing to apply, only a work in progress to go further... One thing you can see is how I propagate the token sources using: 137: n-&gt;from = list-&gt;pos; ... 143: list-&gt;pos.line = id; 144: list-&gt;pos.stream = pps; here you get an argument why it is better to have a (1) -&gt;macro_begin(a) -&gt;macro_end(b) instead of only one (2) expand_macro(a,b) If you want to use the preprocessorhooks to output human readable macro expansion history line similar to LLVM you probably want a pointer to the "pre-expanded" token location, that is in (a). In (1) you can buildup the token-source-paths going from post-expand buffer (b) through the pre-expanded buffers (a) in (2) you only have the paths going through the expanded buffers, can use (a) to reason about where (b) came from, but not in a simple way... Kindof something like this: #define E1 #define E2 #define S1(a) struct a { E1 int d1; }; #define S2(a) struct a { E2 int d2; }; #define </pre> Konrad Eisele 2012-05-15T07:52:23 http://permalink.gmane.org/gmane.comp.parsers.sparse/2855 <pre> I am very sorry that my speed of absorbing patches is much slower than your speed of producing it :( About propagating the empty expansions, may I ask a silly question? Is that the goal is you are able to track that, after the recursive expansion, you are able to tell in the result stream, there is an macro expand to nothing in the this location? Obviously, if the empty expansion is happen in the top level macro expand, you can always keep track of where is the original location of the expand using macro token-&gt;pos. The tricky part is the, if the empty expansion happen inside a multi-level macro expand. Then it is hard to keep track of where that empty macro should have been landed if it is not empty. Is that the problem you are trying to solve? About the two example usage you give. The first one is using the html to show how macro expand recursively. I like that demo. It only need to remember at which level the macro expand to empty. It don't need to remember where the empty macro need to land in the re</pre> Christopher Li 2012-05-15T06:30:20 http://permalink.gmane.org/gmane.comp.parsers.sparse/2854 <pre> Here is some feed back for the patch you send out: The test file should be in validation/ directory. Please give a.h a name represent the test as well. + if (!strncmp(arg, "buildin", 7)) { + fnobuildin = 1; + } Is there a stand gcc option to handle nobuildin? I haven't found one. If gcc has one, we can duplicate the gcc behavior. Otherwise, I don't see it is necessary to add one. The dependence program needs to able to handle the symbol from built in stream any way. If it is for easier to debug, your program can trivially skip out the builtin. stream. + * BSD-License + * Redistribution and use in source and binary forms are permitted Can you make it cover by the sparse license file as well? I don't mind you grant extra license to your code. But please at least make it cover by the license of the project itself otherwise I won't apply it. It is pure overhead to figure out what license is compatiable to spase and maintain different file at different lice</pre> Christopher Li 2012-05-14T10:53:35 http://permalink.gmane.org/gmane.comp.parsers.sparse/2853 <pre> I have thought about how to implement empty expansion tracing without introducing a new token type. I came up with a solution, however I need one callback, I called it substitute_arg(), see patch attached. What do you think, is it apply-able? I think I can use the address of the pointer to token (strict token **, which is normally &amp;tok-&gt;next) as a hashing to propagate the empty expansions... Im not 100% shure it works but I need the extra hook to be able to propagate the empty expansion from the arguments into the substitution body... diff --git a/pre-process.c b/pre-process.c index fb3430a..73a58be 100644 --- a/pre-process.c +++ b/pre-process.c &lt; at &gt;&lt; at &gt; -573,6 +573,9 &lt; at &gt;&lt; at &gt; static struct token **substitute(struct token **list, struct token *body, struct case TOKEN_MACRO_ARGUMENT: arg = args[body-&gt;argnum].expanded; count = &amp;args[body-&gt;argnum].n_normal; +if (preprocess_hook) { +preprocess_hook-&gt;substitute_arg (&amp;added, &amp;args[body-&gt;argnum].expanded); +} if (eof_token(arg)) { state = N</pre> Konrad Eisele 2012-05-13T08:52:21 http://permalink.gmane.org/gmane.comp.parsers.sparse/2852 <pre> To explain mdep.c: There are in fact only 3 lines that are of interest: ... 137: n-&gt;from = list-&gt;pos; ... ... 143: list-&gt;pos.line = id; 144: list-&gt;pos.stream = pps; ... Line 137 saves the last token.pos , (143+144) insert a new id into token.pos. This will generate the path for each token through the expansions. mdep_trace() traverses the path... -- To unsubscribe from this list: send the line "unsubscribe linux-sparse" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Konrad Eisele 2012-05-12T17:57:16 http://permalink.gmane.org/gmane.comp.parsers.sparse/2851 <pre> Appended is a test-patch that adds test-mdep testcase. The file mdep.c is used to record that macro expansion, each token will have a reference to its source. test-mdep.c does pre-process (as test-macro.c) then prints out the token trace through macros for each token: &lt; at &gt;{ } is used to mark the active path. An example file is added: a.h $test-mdep a.h ... 0004: 8 body in D1 :4 &lt; at &gt;{8} 10 9 5 &lt;untaint: D1&gt; arg0 in D1 :&lt; at &gt;{8} 10 9 body in D0 :1 &lt; at &gt;{D1}(8 10 9) 2 D2(11) 3 &lt;untaint: D0&gt; a.h:6:6 ... Token nr 4 of the preprocess stream is "8". The generation path of "8" is marked &lt; at &gt;{8}... Not 100%, still, I think already readable. (Actually the printout order should be reversed (starting from file scope and drilling down the macro expansions...) I still dont handle empty expansions. I'll see weather I can come up with something here... No, only a linked list that model the nexting levels. Then a preprocessor hook that can register lookup_macro() macro lookups inside # preprocessor lines. An ex</pre> Konrad Eisele 2012-05-12T17:46:48 http://permalink.gmane.org/gmane.comp.parsers.sparse/2850 <pre> The expand_macro call back use the parent argument which get from expanding_macro list. The caller should be able to create tree from the leaf node using the parent pointer. Feel free to change to use the expanding_macro instead if that make building the tree easier. The body expansion can't be recursive on same macro otherwise it can result in unlimited expansion. The C stander specify the macro expand this way. No problem at all. I figure you just want to the patch to get included. That is great. I have a test-macro program in that branch which is very close to print out all the tokens. Let me think about this. Just thinking out lound, The #include and #ifdef can consider as a special kind of predefine macro as well. For symbol, it relative easy because symbol has pos range and aux pointer. Do you need to attach the dependency for the statment and expression as well? Chris -- To unsubscribe from this list: send the line "unsubscribe linux-sparse" in the body of a message to majordomo&lt; at &gt;vger.ke</pre> Christopher Li 2012-05-12T11:02:17 http://permalink.gmane.org/gmane.comp.parsers.sparse/2849 <pre> This seems ok. expanding_macro has to be global not static to be used... (?) I think the fact that argument expansion is recursive and body expansion is non-recursive is one of the things that make the preprocessor kindof hard to grasp. I cannot say this before I've tried it. I'd like to straighten things out a bit: My last emails where a bit too harsh and I'd like to apologize. Sorry for that. The next step then is: I'll write a patch to add a test-prog that uses this api to trace the token generation and generate a tree for it. For a start I'll printout for all tokens of a preprocessor run all macros-expansions that generated them. Now, I've learned not to run too fast towards the goal, (which is still "dependency tee from c parser entities downto token"), maybe you can think about how to achieve the next steps in an API : - An #include #ifdef #else #endif pushdown-stack to record the nestings for each token - How to connect all this to the AST. </pre> Konrad Eisele 2012-05-11T21:48:27 http://permalink.gmane.org/gmane.comp.parsers.sparse/2848 <pre> You have a valid point that B(B(x)) will break the sym-&gt;parent list. I remove the sym-&gt;parent and just use a token_list to maintain the expanding macro. The macro is append to the list when enter and remove from the list when untaint token is reached. The change is in the same review branch. That should solve this problem? That is right. The macro expansion has 3 stages. The first stage is expand the arguments list while the caller macro does not consider "tainted" during the argument expansion. The macro can recursively appear in the argument list. That is some thing I haven't consider previously. The second stage is just replace the expanded arguments into body. The third stage is rescan the replacement string for macro expand. In this third stage, the macro itself is consider tainted and can't be expand again during the rescan. Can you take a look at this modify version will fit your need or not? I am curious the part weather that will remove the need to add empty token to the list for expansion. </pre> Christopher Li 2012-05-11T19:40:57 http://permalink.gmane.org/gmane.comp.parsers.sparse/2847 <pre> I've found lkml.org somewhat unreliable. I'd suggest liking to http://mid.gmane.org/&lt;the-message-ID&gt; ; in this case, http://mid.gmane.org/CA+55aFzbhYvw7Am9EYgatpjTknBFm9eq+3jBWQHkSCUpnb3HRQ&lt; at &gt;mail.gmail.com - Josh Triplett -- To unsubscribe from this list: send the line "unsubscribe linux-sparse" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Josh Triplett 2012-05-10T18:24:40 http://permalink.gmane.org/gmane.comp.parsers.sparse/2846 <pre> I have to revise my previous assumption though: #define B(y) A(x) B(1) i thought that in the body expansion of B recursion is involved, so that it would yield: expand_macro(A); expand_macro(B); but that was wrong, so its expand_macro(B); expand_macro(A); you might be right here. expand() is flat when substituting the body part of the macro. It might work. </pre> Konrad Eisele 2012-05-10T12:28:23 http://permalink.gmane.org/gmane.comp.parsers.sparse/2845 <pre> As far as I can tell, the type can be recovered reliably, one way or the other, and missing pointer cast was one of a few (if not only one) source of type loss in the instruction stream (e.g., with those pointers returned by function call as demonstrated). Thanks. </pre> Jan Pokorný 2012-05-10T12:20:49 http://permalink.gmane.org/gmane.comp.parsers.sparse/2844 <pre> Do a B(B(x)) and your sym-&gt;parent linked-list will fail. </pre> Konrad Eisele 2012-05-10T12:14:42 http://permalink.gmane.org/gmane.comp.parsers.sparse/2843 <pre> CCing Linus, Not sure about the rules of quoting other people's email. Does it need the full copyright notice? I would just add these two lines to the end of the sparse.txt Reference: * Linus' email about __nocast vs __bitwise: https://lkml.org/lkml/2012/3/22/409 Chris -- To unsubscribe from this list: send the line "unsubscribe linux-sparse" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Christopher Li 2012-05-10T11:42:22 http://permalink.gmane.org/gmane.comp.parsers.sparse/2842 <pre> Well, the __cannot__ part is base on your reply you seems don't wish to continue this discussion. A change like this is bound to need some careful discussion and planing. Yes, I am guilt of only accepting patches meet some subjective stander of mine. But so is to any self respect project maintainers. I would rather spend some time to do it right than commit some thing I would regret later on. I heard you that this discussion is taking long. That is why I offer to write up the core sparse part of the change myself and let you provide feed back to shape it the way we both can happy. That is the agreement we have earlier right? So I did exactly what I said I am going to do, now you are calling me my way vs your way? My evaluation function is straightly technical merit: - I prefer patch minimize performance impact on other clients don't use this feature. - I prefer simpler interface over complicate one. To me, believe it or not, It is never about my way vs your way. If you submit a perfect patch, I would </pre> Christopher Li 2012-05-10T11:25:05 http://permalink.gmane.org/gmane.comp.parsers.sparse/2841 <pre>Hi Chris: --- On Thu, May 10, 2012 at 4:20 PM, Christopher Li &lt;sparse&lt; at &gt;chrisli.org&gt; wrote: | Looks good. Thanks for doing that. | | Can you add some reference (in sparse.txt) to the original Linus email | where some | of this document is base on? Just to give credit where it is due. A link to the | Linus original email would be perfect. \-- Is the following reference sufficient if added to the sparse.txt file? === sparse.txt === Sparse ~~~~~ Copyright (C) 2012 Linus Torvalds Source: https://lkml.org/lkml/2012/3/22/409 __nocast vs __bitwise: ... ... ... === END === I will re-submit the patch with the approved changes. Thanks for your quick response! SK </pre> Shakthi Kannan 2012-05-10T11:06:28 http://permalink.gmane.org/gmane.comp.parsers.sparse/2840 <pre>Looks good. Thanks for doing that. Can you add some reference (in sparse.txt) to the original Linus email where some of this document is base on? Just to give credit where it is due. A link to the Linus original email would be perfect. Thanks Chris On Wed, May 9, 2012 at 11:54 PM, Shakthi Kannan &lt;shakthimaan&lt; at &gt;gmail.com&gt; wrote: -- To unsubscribe from this list: send the line "unsubscribe linux-sparse" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Christopher Li 2012-05-10T10:50:26 http://permalink.gmane.org/gmane.comp.parsers.sparse/2839 <pre> The early sparse back end does not care about preserve the ctype in the back end. The type has been simplify to bit size. So the pointer type conversion which has the same bit size, it does not need to generate any back end machine code e.g. x86 instruction to perform the conversion. I think that is why it is skipped. The recent changes, especially the llvm back end care more about preserving the ctype in the back end instruction level. The patch is applied. Chris -- To unsubscribe from this list: send the line "unsubscribe linux-sparse" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Christopher Li 2012-05-10T10:42:59 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.parsers.sparse