gmane.comp.security.openxpki.devel

Development tracks

Martin Bartosch — 2012-05-24T19:37:00

Hi folks,

I like the momentum Oxi has been getting lately - possibly due to moving from SVN to git, which allows us to pursue experimental and potentially disruptive development.

The develop branch will soon see the integrating of the following development tracks. This will probably result in a huge construction site, but - hey - it's the develop branch for a reason ;-)

- database unification and improvements (Michael's submission for indexing LOBs) -> schema change necessary
- Dieter's submission to the WF engine (paused workflows) -> schema change necessary
- watchdog daemon keeping track of stalled workflows
- inclusion of the NICE API (Oliver)
- inclusion of the new Connector infrastructure we are currently working on (and successive migration of the core configuration to Connector)

Some things to come later this year:
- redesign of the SCEP workflow: many more configuration options, allowing for flexible policies for renewal and support for initial enrollment via SCEP (challenge password)
- inclusion of the current Smartcard personalization development: a complete, user friendly and highly customizable Smartcard personalization and certificate life cycle management application
- multi node and cluster support (active-active setup with shared database)
- a VMWare appliance as a showcase for Oxi (hopefully including the new Smartcard personalization with all required external dependencies, such as mock LDAP and Active Directory servers)

cu

Martin


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Re: paused workflows,proc state and more: first implementation

Martin Bartosch — 2012-05-24T19:20:41

Hi Dieter,


first of all: welcome to the team. I am glad to hear that the paused workflow implementation is already implemented. I will have a look once I've got a bit less pressure in my current project - hopefully soon (see my other mail).

cheers

Martin


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Re: exploding load after search queries

Martin Bartosch — 2012-05-24T19:19:45

Hi Michael,


apologies for not following up, I have not forgotten this - I was just too busy lately to have a look. I hope I will have a chance to do this in about 2 weeks.

cu

Martin


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Re: paused workflows, proc state and more: first implementation

Dieter Siebeck — 2012-05-23T09:11:26

UPDATE:

I now have added a suite of 4 QA tests, which give a good impression of 
how the new workflow features work.

view source here:
https://github.com/dsiebeck/openxpki/tree/feature%2Fworkflow


1) normal execution without pause/exception

perl 10_run_without_pause.t
1..12
ok 1 - Connect to server
ok 2 - Create test workflow
ok 3 - Expecting state STEP2
ok 4 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 5 - Expecting state STEP3
ok 6 - Proc-State should be "manual"
ok 7 - count try should be "0"
ok 8 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 9 - Expecting state SUCCESS
ok 10 - Proc-State should be "finished"
ok 11 - count try should be "0"
ok 12 - wakeup should be empty

2) execution with pause and wake up

perl 11_run_wit_pause.t
1..22
ok 1 - Connect to server
ok 2 - Create test workflow
ok 3 - Expecting state STEP2
ok 4 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 5 - Paused (iteration 1): state should remain STEP2
ok 6 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 7 - Paused (iteration 2): state should remain STEP2
ok 8 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 9 - Paused (iteration 3): state should remain STEP2
ok 10 - cause of pause should be saved in wf context
ok 11 - wake_up was called 2 times
ok 12 - Proc-State should be "pause"
ok 13 - count try should be "3"
ok 14 - wakeup should not be empty
ok 15 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 16 - execute again without pause: state should be STEP3
ok 17 - Proc-State should be "manual"
ok 18 - count try should be "0"
ok 19 - wakeup should be empty
ok 20 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 21 - final state should be SUCCESS
ok 22 - Proc-State should be "finished"

3) maximal allowed retries exceeded:

perl 12_count_try_exceeded.t
1..18
ok 1 - Connect to server
ok 2 - Create test workflow
ok 3 - Expecting state STEP2
ok 4 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 5 - Paused (iteration 1): state should remain STEP2
ok 6 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 7 - Paused (iteration 2): state should remain STEP2
ok 8 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 9 - Paused (iteration 3): state should remain STEP2
ok 10 - 4th execution should result in exception
ok 11 - need correct error exception
ok 12 - exception code should be saved in wf context
ok 13 - State after exception must still be STEP2
ok 14 - Proc-State should be "retry_exceeded"
ok 15 - resumed execution after exception
ok 16 - Action::resume() was called
ok 17 - Proc-State should be "manual"
ok 18 - count try should be "0"

4) "unexpected" exception while execution of action

perl 15_workflow_exception.t
1..13
ok 1 - Connect to server
ok 2 - Create test workflow
ok 3 - Expecting state STEP2
ok 4 - execution should provoke exception
ok 5 - need correct error exception
ok 6 - exception code should be saved in wf context
ok 7 - State after exception must still be STEP2
ok 8 - Proc-State should be "exception"
ok 9 - resumed execution after exception
ok 10 - Action::resume() was called
ok 11 - Proc-State should be "manual"
ok 12 - State after exception should be STEP3
ok 13 - exception code in wf context should be empty again


This is the definition of the test workflow:
<workflow>
<type>I18N_OPENXPKI_WF_TYPE_TESTING</type>
...
<state name="STEP1" autorun="yes">
<description></description>
<action name="I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST"
         resulting_state="STEP2">
</action>
</state>

<state name="STEP2">
<description></description>
<action name="I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST"
         resulting_state="STEP3"
         retry_count="3" retry_interval="+0000000015">
</action>
</state>
...
</workflow>

As one sees here, the maximal allowed count of retries is 3. This 
overwrites the definition in "activity.xml":

<action name="I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST"
      class="OpenXPKI::Server::Workflow::Activity::WorkflowTest"
      retry_count="5" retry_interval="+0000000005">
<field name="cause"/>
<field name="action"/>
</action>


Have a nice day!
Dieter

Am 18.05.2012 15:20, schrieb Dieter Siebeck:
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
OpenXPKI-devel mailing list
OpenXPKI-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/openxpki-devel

paused workflows,proc state and more: first implementation

Dieter Siebeck — 2012-05-18T13:20:08

Hi,

my name is Dieter Siebeck and I am quite new to the OpenXPKI project. I 
am working with Oli Welter since nearly 3 months now, mainly on his 
idea/draft of "paused workflows" (see his mail to the list from 2012-02-14).

Now I have published my first implementation of this feature. You can 
see and download it here:

https://github.com/dsiebeck/openxpki/tree/feature%2Fworkflow

(this branch has been rebased on the devel branch 22 days ago.)

POD  Documentation  of most of the new methods is still missing, but I 
wanted to publish the current state, to give other interested people the 
opportunity to see and maybe correct the big picture.
Also missing are test files/scripts. I would appreciate some 
recommendation, which place to look for the most up2date state-of-the 
art testing files for this.

Now the idea of the implementation: basic need was to gain relatively 
low-level control to the execution of workflows and activities. To 
achieve this, the best (and maybe only) way seemed, to install an "own" 
object  of class OpenXPKI::Server::Workflow, which replaces (and of 
course extends) the CPAN Workflow class (the file existed already, but 
only for documentation purposes). Now its a matter of fact, that the 
CPAN workflow Module isn't *really* designed for extendability. You have 
no way to tell Workflow::Factory that you want "My::Workflow" instead of 
"Workflow". And the Workflow class itself has not many access- or hook 
points  to control the execution of code.(for example, in method 
Workflow::set(), the name of workflow-class is hardcoded as the one and 
only class, which is allowed to use the private set-Method).

So I had to use some perl tricks to get around this limitations (if we 
dont want to copy the hole Workflow-Module and extend it directly):

1) OpenXPKI::Workflow::Factory returns always an object of class 
OpenXPKI::Server::Workflow. This is achieved via a friendly capturing 
maneuver:
# OpenXPKI::Workflow::Factory
sub fetch_workflow {
     my ( $self, $wf_type, $wf_id ) = < at >_;
     my $wf = $self->SUPER::fetch_workflow($wf_type, $wf_id);
    ...
    my $oxiWf = OpenXPKI::Server::Workflow->new($wf);
    return $oxiWf;

#OpenXPKI::Server::Workflow
use base qw( Workflow )
sub new{
     my ( $class, $BaseWorkflow ) = < at >_;
     my $self = bless {}, $class;
     #take over all properties from (original) base workflow
     while ( my ( $key, $val ) = each %{$BaseWorkflow} ) {
         $self->{$key} = $val;
     }
     ...
    return $self;
}

not the most beautiful code at all, but - thanks to perls open oop 
structure - it works. OpenXPKI::Server::Workflow has all methods and 
properties from the original workflow.

Another limitation: Workflow::Factory::fetch_workflow() uses only 
certain data (two hardcoded keys) of the returned db data hash of the 
persister.
Since there is no entry point/hook to alter this behaviour, a second 
DB-fetch is needed to retrieve other data from the workflow-table (and 
pass it to the workflow-object):

     my $persister = $self->get_persister( $wf_config->{persister} );
     my $wf_info   = $persister->fetch_workflow($wf_id);
     my $oxiWf = OpenXPKI::Server::Workflow->new($wf,$self,$wf_info);

So the full signature of the worklow constructor is:
     my ( $class, $BaseWorkflow, $Factory, $wfData ) = < at >_;
(The reference of the factory is used to call the 
Factory::save_workflow() method several time to store the current states)

2) OpenXPKI::Server::Workflow overrides the execute_action() - method. 
As a wrapper method, it executes some code before and after 
SUPER::execute_action(). Here is the place to determine/set the 
appropriate "proc state" (which is a new db field and can have the values :

  * init
  * wakeup
  * resume
  * running
  * manual
  * finished
  * pause
  * exception
  * retry_exceeded

The first 3 are not expected to see in "normal" circumstances - because 
they are quite volatile by design. "running" will be set, before 
SUPER::execute_action/Activity::run is called. After eexecution of one 
or more Activities, either "manual" (waiting for interaction)  or 
"finished" will be set.
If an exception occurs, the proc state "exception" is set. Also the 
message code (not translation) will be saved in WF context (key 
"wf_exception")

The two states "pause" and "retry_exceeded" concern the new "pause" feature.

3) OpenXPKI::Server::Workflow::Activity has some new methods and utility 
features:

first the pause() method: every derived concrete activity-class can now 
(somewhere in its "run"-method call:

$self->pause('some cause description');

This will immediately end the execution of the "run"-method (this is 
achieved via OpenXPKI::Server::Workflow::Pause, an "pseudo" exception 
class) .

Additionally, $workflow->pause() is called, which does the hole work: 
check the count of allowed retries, sets the "wake up" timestamp, 
notifiy observers, write history entries etc. The workflow state remains 
unchained (this is achieved via overriding the 
Workflow::_get_next_state() method). If "max_retry" is exceeded, an 
special exception will be thrown, which results in the special proc 
state "retry_exceeded". The given "cause" of the pause will also be 
stored in workflow context, key "wf_pause_msg".

For convenience, the Activity objects can now always "speak" to their 
workflow via $self->{CURRENT_WORKFLOW}. Also I have a new utility method 
"_get_wf_action_param($key)", which retrieves the param $key for the 
current action in the xml-conf of current workflow.

4) Wake up: if an paused workflow is executed again (via command line, 
the new "watcher" demon etc ), the proc state changes to "wakeup" and  
the hook method Activity::wake_up() will be called. Afterwards, 
proc-state changes to "running" and the normal execution begins.

5) Resume: if a workflow with proc state "exception" is executed again, 
the proc state "resume" is set and the corresponing hook method is 
called. Afterwards, proc-state changes to "running" and the normal 
execution beginns.

6) The settings of "count try" and "resume_at" can be set/configured (in 
this order)
manual set via $self->set_count_try/max_retry()
in action definition of the xml config of current workflow
in the xml config of the activity

Additionally, the retry-intervall can be set as second parameter of 
$self->pause('cause', $retry_interval), which has the highest priority.

These are the main points of the implementation. Any existing Activity 
class should run without any modifications and problems.

Of course, the workflow table has obtained some new fields: 
workflow_proc_state, workflow_wakeup_at, workflow_count_try and 
workflow_reap_at (the later corresponds to the "reap at"-Feature, which 
is not implemented yet. Here's the full layout of my workflow table:

CREATE TABLE IF NOT EXISTS `workflow` (
   `workflow_id` decimal(49,0) NOT NULL,
   `pki_realm` varchar(255) DEFAULT NULL,
   `workflow_type` varchar(255) DEFAULT NULL,
   `workflow_version_id` decimal(49,0) DEFAULT NULL,
   `workflow_state` varchar(255) DEFAULT NULL,
   `workflow_last_update` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP 
ON UPDATE CURRENT_TIMESTAMP,
   `workflow_proc_state` varchar(20) DEFAULT NULL,
   `workflow_wakeup_at` timestamp NULL DEFAULT NULL,
   `workflow_count_try` int(11) DEFAULT NULL,
   `workflow_reap_at` timestamp NULL DEFAULT NULL,
   PRIMARY KEY (`workflow_id`)
);


Next steps are:

 1. documentation
 2. write some tests (suggestions welcome, see above)
 3. setting the "reap at" timestamp (as proposed by oli)
 4. implementing the "watcher" process, which runs in background and
    periodically checks, if it finds some paused workflows to "wake up".

Any comments, suggestions and corrections are highly appreciated!

Another question: where would be the best place to put an explanation 
like the one in this email here? In one perl class? Or is there an extra 
place for such docs, which give an overview over the interplay of 
several classes?

have a nice weekend,

Dieter

PS:

Since test files are missing, this is the way to test the feature 
(relying on I18N_OPENXPKI_WF_TYPE_TESTING 
(OpenXPKI::Server::Workflow::Activity::WorkflowTest):

openxpkictl restart --debug 
OpenXPKI::Server::Workflow:128,OpenXPKI::Server::Workflow::Activity:20

openxpkicmd   I18N_OPENXPKI_WF_TYPE_TESTING (expected output: "Workflow 
created (ID: xxxxx), State: STEP2")

openxpkicmd  --wfid xxxxx --param action=pause  --wfaction 
I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST I18N_OPENXPKI_WF_TYPE_TESTING 
(expected output: "New Workflow State: STEP2")


execute last line more than 4 times, and the "max retry exceeded" event 
occurs.

Alternatively, you can set "--param action=crash" to provoke an exception.
to continue normally (till SUCCES) you need to give an arbitrary value 
for "action", e.g. "abc" (due to some limitation in the parameter 
parsing here)












------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
OpenXPKI-devel mailing list
OpenXPKI-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/openxpki-devel

Re: exploding load after search queries

Michael Bell — 2012-05-08T11:19:36

Hi Oli,

Am 05.05.2012 08:08, schrieb Oliver Welter:

I implemented a fix but this fix needs a review by Martin because it
changes the database schema.

https://github.com/bellmich/openxpki/compare/develop...feature%2Fdbi_indexing_lob_columns

Best regards

Michael

Re: Removal of obsoleted database entries

Martin Bartosch — 2012-05-06T14:45:47

Hi Joachim,


We are having a similar issue with lots of SCEP requests. I have documented our solution onhttp://wiki.openxpki.org/index.php/Best_Practices/Housekeeping/Cleanup_Superfluous_Workflows


Yep, something to address in the future :)

cu

Martin


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Re: exploding load after search queries

Joachim Astel — 2012-05-05T12:40:12

I second.

[Cert Attributes]

Ah, i see.

Greetings
   -Achim

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Re: Removal of obsoleted database entries

Oliver Welter — 2012-05-05T12:33:39

Hi Achim

Am 03.05.2012 12:50, schrieb Joachim Astel:

At least not on the "official" sources - but I guess that you can solve 
most of the "garbage collection" with proper relations and triggers in 
mysql.

Oli

Re: exploding load after search queries

Oliver Welter — 2012-05-05T12:31:11

Hi Achim,


Personally, I see the context as an opaque datastore and dislike the 
idea of doing any search queries on the data. I know that we do this at 
the moment in some places but I would like to see this "searchable 
context data" in a dedicated mapping/index/search - whatever you would 
name it - table. In return, we would not need to index the value portion 
of the context table, as it is never used to search within.


If you use subject alternative names or forced validity, this info is 
recorded here as a 1:n relation.

Oli

Re: exploding load after search queries

Joachim Astel — 2012-05-05T10:31:37

Hi Oliver, Martin,

my MySQL implementation allowed the following index creations:

CREATE INDEX wf_state_state on workflow (workflow_state);
CREATE INDEX cert_realm_index on certificate (pki_realm);
CREATE INDEX csr_profile_index on csr (profile(200));
CREATE INDEX csr_role_index on csr (role(200));
CREATE INDEX cert_csrid_index on certificate (req_key);
CREATE INDEX wf_realm_index on workflow (pki_realm);
CREATE INDEX csr_subject_index on csr (subject);
CREATE INDEX cert_role_index on certificate (role(200));
CREATE INDEX wf_hist_wfserial_index on workflow_history (workflow_id);
CREATE INDEX wf_context_key_index on workflow_context (workflow_context_key);
CREATE INDEX wf_type_index on workflow (workflow_type);
CREATE INDEX cert_subject_index on certificate (subject);
CREATE INDEX cert_identifier_index on certificate (identifier);
CREATE INDEX cert_status_index on certificate (status(200));
CREATE INDEX cert_attributes_key_index on certificate_attributes attribute_contentkey);

As you can see i respected your suggestions with 200 bytes field lengths
for the text fields "profile", "role" and "status".

Unfortunately we would also need the solution for workflow_context.
I don't have the experience how large it could grow, i just imagine
large key lengths of 8192 and corresponding blown up certificate lenghts.
This could be rather dynamic in the future, and the database indexes
should correspond to this.

CREATE INDEX wf_context_value_index on workflow_context workflow_context_value);
CREATE INDEX cert_attributes_value_index on certificate_attributes attribute_value);

Cert Attributes don't matter at my setups -- the complete table is empty.
For what is it used anyway within other implementations?

Greetings
    -Achim

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Re: exploding load after search queries

Oliver Welter — 2012-05-05T06:08:43

Hi Achim,

On 04.05.2012 22:47, Joachim Astel wrote:
The fast way - create the indexes on the first 200 chars (mysql offers 
that option). I assume that it is a safe assumption, that index searches 
are always done on scalars which wont be longer than that.

It should be also safe to shrink the role/profile/status fields to 
something like tinytext or varchar 255 - the default roles and status 
names are around are around 10 chars long, the longest profile name is 
49 chars, as long as you did not choose to make very bulky names, that 
should do the trick.

As the problem with the context_value column is even worse on oracle, a proper solution is on the to-do list - so if anybody has some better ideas, just shoot!

Oli

Re: exploding load after search queries

Joachim Astel — 2012-05-04T20:47:35

Hi OpenXPKI-folks,


I've tried this with MySQL as backend database, and ran into the
following index creation problems:

CREATE INDEX csr_profile_index on csr (profile);
-> ERROR 1170 (42000) at line 1: BLOB/TEXT column 'profile' used in key specification without a key length
(profile is a "text" field without length restriction.)

CREATE INDEX csr_role_index on csr (role);
-> ERROR 1170 (42000) at line 1: BLOB/TEXT column 'role' used in key specification without a key length
(role is a "text" field without length restriction.)

CREATE INDEX wf_context_value_index on workflow_context (workflow_context_value);
-> ERROR 1170 (42000) at line 1: BLOB/TEXT column 'workflow_context_value' used in key specification without a key length
(workflow_context_value is a "text" field without length restriction.)

CREATE INDEX cert_role_index on certificate (role);
-> ERROR 1170 (42000) at line 1: BLOB/TEXT column 'role' used in key specification without a key length
(role is a "text" field without length restriction.)

CREATE INDEX cert_status_index on certificate (status);
-> ERROR 1170 (42000) at line 1: BLOB/TEXT column 'status' used in key specification without a key length
(status is a "text" field without length restriction.)

CREATE INDEX cert_attributes_value_index on certificate_attributes (attribute_value);
-> ERROR 1170 (42000) at line 1: BLOB/TEXT column 'attribute_value' used in key specification without a key length
(attribute_value is a "longtext" field without length restriction.)


Is there a way workarounding this MySQL behaviour in not indexing
non-specified field lenghts of TEXT and LONGTEXT fields? > 

Greetings
    Achim

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Removal of obsoleted database entries

Joachim Astel — 2012-05-03T10:50:55

We're doing some database cleanup within our database via cronjob for
our OpenXPKI databases, especially for the audittrail log:

DELETE FROM audittrail WHERE DATEDIFF(NOW(),CONVERT(logtimestamp,DATETIME))>365

Unfortunately, generated CRL (via openxpkicmd) are growing within our
workflow table. We would like to remove old workflows with workflow_type
CRL_ISSUANCE, but backend database entries which are associated with it
would also have to be deleted -- to avoid database garbage.
Are there already scripts around in other projects addressing this issue?

Greetings
    -Achim

PS: I've already deleted workflows with 
"DELETE FROM WORKFLOW WHERE WORKFLOW_TYPE='I18N_OPENXPKI_WF_TYPE_CRL_ISSUANCE'"
I suppose a garbage collector would be good for this problem too. :-)

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Re: SQL DYNAMIC patch incomplete ?

Michael Bell — 2012-05-03T08:39:58

Hi,

after a small private discussion Oliver fixed my bug and implemented
support for the old behaviour ('key' => 'value'). If you use the old
behaviour then you get a warning because you use a deprecated part of
the code.

The old behaviour is used to avoid any breaking of the code.
Additionally we think that there should only be one correct/right way.
So I would like to remove the old behaviour in terms of clarity and
simplicity. Oliver immediately discovered the backside of my straight
forward approach. Finally we agreed in a compromise - support for the
old behaviour with a big warning.

Best regards

Michael

Am 26.04.2012 19:24, schrieb Oliver Welter:

Re: SQL DYNAMIC patch incomplete ?

Michael Bell — 2012-05-03T08:38:29

Am 26.04.2012 22:45, schrieb Michael Bell:

This was in fact the problem (plus an additional bug in the new code
from Oliver).


The patch was incomplete.

Best regards

Michael

Re: SQL DYNAMIC patch incomplete ?

Oliver Welter — 2012-04-27T10:37:08

Hi Micha,


On 26.04.2012 22:45, Michael Bell wrote:
That should be fixed with 710da93cd.

I tried to get a "fallback to old syntax" into the code - might you 
please do me a favour and have a look on that.
Its commit 23717d68, which is currently only in my private repo.

Oli

Re: SQL DYNAMIC patch incomplete ?

Michael Bell — 2012-04-26T20:45:37

Hi Oli,

Am 26.04.2012 19:24, schrieb Oliver Welter:

DATAPOOL_KEY is still in wrong format.

Best regards, Michael

P.S. Yes, it looks like the patch was incomplete.

SQL DYNAMIC patch incomplete ?

Oliver Welter — 2012-04-26T17:24:55

Hi Micha,

I am running into some problems using the datapool after merging your 
dynamic-patch.

my $msg = CTX('api')->get_data_pool_entry( {
   KEY => $chipid ,
   NAMESPACE => 'smartcard.smartchipid'
} );

results in an exception (I added ref and column)

'LABEL' => 
'I18N_OPENXPKI_SERVER_DBI_SQL_SELECT_DYNAMIC_PARAM_NO_HASH_REFERENCE',
PARAMS' => {
     '__REF__' => '',
    '__COLUMN__' => 'DATAPOOL_KEY',
    '__TABLE__' => 'DATAPOOL'
}

I added some more debug in SQL.pm lin 970

##! 1: ' DYNAMIC arg ' . Dumper $args->{DYNAMIC}

OpenXPKI::Server::DBI::SQL::select (line 970):  DYNAMIC arg $VAR1 = {
           'DATAPOOL_KEY' => 'chipid1234',
           'NAMESPACE' => 'smartcard.smartchipid',
           'PKI_REALM' => 'I18N_OPENXPKI_DEPLOYMENT_TEST_DUMMY_CA'
         };


I found the  {value => xx } brackets are missing in the statement in 
Server/API/Object.pm, Line 604ff , but even after adding them, the error 
remains the samem, having this debug output:

OpenXPKI::Server::DBI::SQL::select (line 970):  DYNAMIC arg $VAR1 = {
           'DATAPOOL_KEY' => 'chipid1234',
           'NAMESPACE' => {
                            'VALUE' => 'smartcard.smartchipid'
                          },
           'PKI_REALM' => {
                            'VALUE' => 
'I18N_OPENXPKI_DEPLOYMENT_TEST_DUMMY_CA'
                          }
         };


I dont know what I am overlooking but I cant get it to work =(

Oli

Re: Git and OpenXPKI … a short primer for people new to git and GitHub

Andreas Leibl — 2012-04-15T17:02:16

Hi Michael,

This is not a GitHub thing ... it's the git way of doing (somewhat) centralised development with a de-centralised version control system.

To git every repository is equal. If we assign more importance to one repo(sitory) than another it is by convention only. Period.

Now if a couple of people work together they could conceivably all have their own version of the product and only pull changes from others as they see fit. Git would support that 100%. The problem is that, as the product evolves (grows), it takes more and more effort to keep in sync for the features/fixes you want.

In real life, it's often more practical to agree on one source repo as a reference ... and then make changes relative to that. You can still have your own version (branch, fork, whatever) but you would be moving alongside this agreed central reference. Take Linux as an example. The rep Linus Torvalds maintains is, from a git point of view, no better than anybody else's. However, since most people agree he's doing a great job at nurturing Linux and integrating changes (almost) everybody is using it as a reference and uses it's version count for starters.

This central repo is what we call "upstream".

In case of OpenXPKI, it's openxpki/openxpki on GitHub. No (GitHub) magic involved.

Does that clarify things? Or am I completely off?

Kind regards

   Andreas

Re: Git and OpenXPKI … a short primer for people new to git and GitHub

Michael Bell — 2012-04-15T15:58:04

Hi Andreas,

github.com uses a remote repo called upstream. The primer document does
not describe the usage of such a repository.

So if I understand the beginner documentation on github.com correctly
then the changes to upstream are never visible in the forked repo. So I
think that the described developer workflow brings the forked repo
permanently out of sync with openxpki/openxpki.

Is my understanding of git correct here?

Best regards

Michael


Am 02.04.2012 10:43, schrieb Andreas Leibl: