Skip to end of metadata
Go to start of metadata

This document lists gaps between the current set of information stored in OSG/OIM, WLCG/GOCDB and information needed to auto generate Meshconfigs.

Background

Goal of this project is to take information sources such as these..

WLCG

I've given pointer for both AGIS and GOCDB to load data for WLCG. I am not sure which ones I need to be using..

(AGIS)

(GOCDB)

OSG

And use that information to generate mesh configs that looks like following

WLCG (Currently updated manually - so I heard)

OSG (Already auto-generated but it's still a prototype)

Both WLCG / OSG information sources are missing some information needed to produce above mesh configs, and goal of this document is to list all such "gaps" and propose on how to store & expose such information.

Gaps

Item 1. Table of published mesh configs and "tests" under each mesh config (& resource membership)

We need to define a list of mesh configs to be generated (for each "area"?) and for each mesh config define a list of "tests" which contains the test parameter and test type ("mesh",  "disjoint", or  "star"). disjoint and star tests can have 2 member lists (A list and B list) (mesh has only 1)

OSGWLCG

"meshconfigs" table should be created to host a list of mesh config with following attributes.

  • meshconfig_id
  • Name of the mesh config (ex. tests-us-atlas)
  • Description of the mesh config (used as mesh config "name"?)
  • Administrators (OIM contact IDs)

"membergroups" table will defines a list of membergroups (but not the actual hosts) with following attributes.

  • membergroup_id
  • Name of the group (ex. "inter-cloud-latency::disjoint group A")
  • type (either bwctl or owamp)

"meshconfigs-tests" table will host a list of "tests" for each mesh config (1-to-many) with following attributes.

  • test type ("mesh",  "disjoint",  "star")
  • description (used as test "name"?)
  • foreign_membergroup_id_A
  • foreign_membergroup_id_B (can be null for mesh type)
  • mesh config parameters

(owamp test sample) 

(bwctl test sample) 

 

OIM will allow resource services to hold "service attributes" key-value pairs (similar to GOCDB's service extension properties?). For perfsonar toollkit service, we will add a new key; "MESHCONFIG_MEMBER_GROUP_IDS" with values set to delimited IDs from "membergroups" table above. Resource administrator for each perfsonar toolkit instance can edit the list of membergroup IDs (using the actual names instead of IDs)

** I will filter membergroups by either bandwidth or latency stored in membergroup table.. We shouldn't be allowing Bandwidth service instance to be member of Latency group as shown in the screenshot..

 

Assuming that it's perfsonar administrator who gets to decide which mesh config to be part of.

Icon

Assuming that it's perfsonar administrator who gets to decide which mesh config member groups to be part of.

User interfaces to allow OSG support staff to define meshconfig / membergroups information will need to be implemented. (Add / Remove /Edit etc..). The list of mesh/tests themselves should be published via MyOSG apart from the actual mesh configs generated.

Same as OSG - unless this information is already available at URL I am not aware of.

Some of these information can be stored as service extension properties (https://wiki.egi.eu/wiki/GOCDB/Input_System_User_Documentation#Extension_Properties) and exposed under /query/get_services API (question) but it's up to GOCDB

Item 2. Organization and Site administrators

Icon

Many WLCG mesh configs currently contains no organizations nor site administrators, although some has organization contacts (sites-nd-all.json).

WLCG mesh configs also seem to be grouped with individual organization entry for each site ("sites" grouping inside mesh config is unused, and each WLCG sites are stored as "hosts").

Both OIM and AGIS provides address / latlng information for each sites (although current WLCG mesh config doesn't include location information).

OSGWLCG

Organization administrator is currently missing, but OIM can provide site and host administrator from OIM support center and resource admin information. I believe OIM has enough contact information to generate a useful mesh config.

I see email address available under site, but not full name. Organization and host administrator information is missing, but I am not sure if they are needed. If site site admin info is enough, then AGIS just have to provide admin contact name.

Following is a sample of site information AGIS. Please notice that only email address is available.

 

Item 3.

(absorbed into Item 1)

Item 4. Primary and Secondary perfsonar toolkit endpoints for each CE and SE

Icon
This item does not directly concern the perfsonar mesh config generations, but since it's similar to other changes I am proposing I am noting it here.

We need to be able to specify which perfsonar toolkit instance should be used to analyze information for each SE/CE (I think we should have some kind of public service that allows user to enter an IP/host and tell user nearest perfsonar instance.. but that's another story).

OSGWLCG

Under OIM/Resource/CE&SE services, we will add a select box allowing administrator to enter primary and secondary perfsonar toolkit instance responsible for monitoring that CE or SE.

The information will be exposed as part of resource service information via MyOSG.

Similar to OIM. The information should be stored as service extension properties and exposed under /query/get_services API (question)

Appendix

OSG mesh

OSG mesh (http://myosg.grid.iu.edu/miscpfmesh?count_sg_1&count_active=on&count_enabled=on) are currently generated using following mapping rules.

  • VO ownership information is used to create different mesh configs.
  • oim/facility == organizations (description) *org admin information is currently missing.
  • oim/site == site (location, description)
  • oim/support center admin == site admin *not sure if this is the right thing to do.
  • oim/resource/service == host
  • oim/resource/admin == host_administrator

Mesh config structure

 The  "green"  text  shows  optional  information  and  the  "red"  text  shows  required  (at  least  by WLCG policy). 

 

  • No labels