RE: Juniper to be added to the official list of CNAs

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

RE: Juniper to be added to the official list of CNAs

jericho
This was originally posted to the 'private' Editorial board list. I am
moving this thread to the public list as well, because it involves the
industry at large. The private list should only be used for matters
related to the board, such as voting on new members, not for discussing
industry-wide issues. Also, please note that the move to private list has
happened more in the last 60 days than it has in the last 6 - 18 months.
This is not acceptable to the industry.



Joe,

On Tue, 19 Apr 2016, Common Vulnerabilities & Exposures wrote:

First, can MITRE quit posting as "Common Vulnerabilities & Exposures
<[hidden email]>" please? There are more than 10 MITRE employees on the
Editorial Board list, that are not members of the Board. I am happy to
enumerate them if there is any question about that fact. This specific
response came after you (Joe) joined the fray too, and your title:

  Joe Sain
  CVE Communications and Outreach Lead

So I have to assume this is you. If I am wrong, it only makes my point for
me.

We need accountability in the face of all the criticism MITRE has received
the last year. It is not ethical, or appropriate that anyone there hide
behind the CVE name. Or "cve-id-change" (one post historically) or
"CVE-assign" (one post historically). This isn't conducive to trust.

From here out, I suggest that MITRE only reply to board traffic from an
individual, even if it is a general 'CVE' policy proposal. The board list
is for discussion of ideas. If the final, voted-on, decision comes from a
generic CVE address, I can see that as a proper use of an alias, maybe.

: Juniper, as a new CNA, will become better over time as they practice
: being a CNA.  Another member suggested that all CNA-related documents be

Wait... they failed to follow CNA guidelines *before* they were a CNA.
Meaning, they asked for assignments from MITRE, who issued them. And
Juniper published advisories that were problematic, and didn't follow CVE
abstraction. MITRE is rewarding them for that behavior, by giving them
full CNA status, saying "they will learn"?

I am officially objecting to this policy and precedent. This is absolutely
the wrong move, and not going to help the mess that is CVE. Worse, you did
so six days after a formal complaint about Juniper, from an active board
member? And... worser(?), you did it 7+ months after I specifically asked,
and hounded MITRE on, providing official CNA guidance documentation. This
is clearly an effort of MITRE to produce more CNAs to help alleviate the
assignment workload, while ignoring many Editorial Board members saying we
need more CNAs over the last three years. Bandaids aren't going to work at
this point, and this is a perfect represenation of such a bandaid. Taking
our advice three years later, without proper documentation, is a
step-by-step recipe for more problems.

Remind me, why are we, the board, here? To expand on this... I have been
the only one that I am aware of, policing several CNAs that are not
following the old legacy guidelines re: abstraction. I have probably filed
more complaints to MITRE on CNAs than anyone else. If that isn't the case,
please introduce me to whoever is doing it more than I am. I'd like to
compare notes. Why? Because I only mail once out of every ~ 25 instances
of a CNA not following rules. e.g. IBM jumped the CNA shark a year or two
ago. When I pointed it out repeatedly, and showed they continually gave
the wrong assignments for known/public issues, the response from MITRE was
"you are right, we MIGHT contact them". To this day, I don't know if MITRE
contacted IBM, but I do know they kept using the same offending assignment
three months after that mail thread. I have to assume MITRE ignored the
rogue CNA, and ignored the complaints from a board member.

At some point, MITRE needs to address these issues publicly. The reason
people are not happy with this situation, and DHS should be fully aware
of, is that most of the solutions were handed to MITRE on a silver platter
all along. Every step of the way, MITRE ignored them.

: posted publicly so that all CNAs understand better what the CNA
: requirements are. This is a good idea and we have established a GitHub
: site for these documents at: http://cveproject.github.io/docs/.  The

I'm sorry, GitHub is generally accepted to be at github.com. Why did MITRE
choose to use github.io, a "GitHub pages" domain that was converted in
2013, that has some fruity integration with github.com (meaning the UX is
is lacking)? Why wasn't that discussed with the board? Why was that site
chosen AFTER the DWF initiative specifically chose GitHub.com due to
prevalence and adoption? Every single belated reaction from MITRE to the
CVE problems are answered by the textbook definition of "worst solution".
When those decisions are questioned, MITRE goes quiet... both on list, and
off list. I have the emails to prove that if you have any doubt.

Could MITRE form a team to figure this out, and work toward providing a
more friendly and intuitive experience for board members bringing up
problems? If you start a random crappy hosted RedMine tracker to track
these issues, I will scream.
Reply | Threaded
Open this post in threaded view
|

RE: Juniper to be added to the official list of CNAs

Common Vulnerabilities & Exposures
Brian -

Thank you for your thoughtful reply.  The CVE Team will continue to post under the name CVE Team.  Where specific points of contact are necessary, the members of the Board will be provided those.  I am the MITRE CVE Project Lead, having assumed this role at the beginning of the year.  If accountability is the concern, my contact information is below.  I'm happy to field any questions, concerns and comments anyone may have on behalf of the CVE Team.
 
Your opinion regarding what should be posted to public versus private lists is valid, but there are others who may have different opinions that are equally valid.  Since all Board members are equal and entitled to their own opinions, all opinions must be considered.  For example, the note to the private Board list yesterday regarding Juniper was intended to provide all Board members with an opportunity to privately voice opinions in a candid fashion that they may have been uncomfortable voicing in public.  In this context, it is the person who posts the opinion who is best able to determine whether they want their opinion posted publicly, and is not up to anyone but them to make that decision if the original intent of a private list is to be honored.  Using the private list does not preclude in any way public discussions, and in many ways can accelerate the tempo and quality of such discussions.  As we collectively rework the roles and interactions of CNAs and the broader issue set around the CVE capability, there will likely be occasions where private discussions are required to better serve what is discussed publicly.
 
We understand and appreciate your objections to Juniper.  Juniper is not being rewarded for anything.  Rather, they are being brought online as a new CNA so that we can expand the CVE capability consistent with the stated objective of our Board colleagues to scale the capability under a federated approach to increase coverage.  We were delighted to hear Juniper's enthusiasm to be active, flexible participants in charting the way forward.  They are best positioned to do this as a CNA, as is Intel.  It gives them a real stake in the outcomes we collectively wish to achieve.  This is the CVE Team's opinion that we look forward to discussing with our Board colleagues.  More broadly, the CVE Team understands the issues with CNAs; such issues have not been ignored and our goal is to actively address them with the Board.  In the past, the CVE Team has not effectively communicated with the Board in terms of frequency, content and follow-through.  We acknowledge this, apologize for it, and intend to make this right going forward.  We voiced this at the 30 March discussion and look forward to the Board call tomorrow to continue the positive trajectory in dealing with the dozens of issues that will arise as we collectively work to scale the capability.  We have adopted the "fail fast" mentality.  That mentality applies to more than just the DWF pilot.
 
I am unsure what "fruity integration" means in the context of GitHub.  We committed to the Board to get our documents up on GitHub at the 30 March discussion.  That is done.  We use the site for other non-CVE projects and have had good experience with it.  We use github.io as a simple way to present the mark down expressed documentation.  Is there a specific issue that underpins "fruity integration" that you are able to make us aware of? If you prefer not to work within the github.io presentation layer, you may access the documents in the "cna" and "content" directories at: https://github.com/CVEProject/docs.
 
The CVE Team is receptive to any means the Board determines is appropriate for effective collaboration.  At the 30 March meeting, GitHub was suggested and we agreed.  We are entirely open to other suggestions and have some of our own.  For example, Google Docs may be a good place to develop first version documents prior to releasing them on GitHub for public review and comment as this allows a smaller group of very knowledgeable experts to establish something that makes sense based on our collective experience, thereby minimizing the transaction costs when we engage the public.  In certain cases, this may be an appropriate approach while in others it may not be.  The Board is best suited to decide these matters on a case-by-case basis.
 
The CVE Team has the following objectives:  1) effectively communicate with CVE stakeholders; 2) improve operational efficiency; and 3) scale the CVE capability.  All of our team members believe in and are accountable for achieving these objectives, which were established in February 2016.  We fully understand that the answers to many of the issues that must be addressed are not resident within our knowledge base.  We reached out to the Board to schedule the 30 March meeting and greatly appreciate their willingness to meet every two weeks on an ongoing basis to better identify issues, structure the decisions required to resolve the issues, and make concrete decisions to move the capability forward.
 
Regards
 
The CVE Team
___________________
Chris Levendis
MITRE
Homeland Security Systems Engineering and
Development Institute (HS SEDI)
(MITRE) 703-983-2801
(Cell)    703-298-8593
[hidden email]

-----Original Message-----
From: jericho [mailto:[hidden email]]
Sent: Wednesday, April 20, 2016 2:16 AM
To: Common Vulnerabilities & Exposures <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: RE: Juniper to be added to the official list of CNAs
Importance: High


This was originally posted to the 'private' Editorial board list. I am moving this thread to the public list as well, because it involves the industry at large. The private list should only be used for matters related to the board, such as voting on new members, not for discussing industry-wide issues. Also, please note that the move to private list has happened more in the last 60 days than it has in the last 6 - 18 months.
This is not acceptable to the industry.
Reply | Threaded
Open this post in threaded view
|

RE: Juniper to be added to the official list of CNAs

jericho
On Wed, 20 Apr 2016, Common Vulnerabilities & Exposures wrote:

: Brian -
:
: to their own opinions, all opinions must be considered.  For example,
: the note to the private Board list yesterday regarding Juniper was
: intended to provide all Board members with an opportunity to privately
: voice opinions in a candid fashion that they may have been uncomfortable
: voicing in public.  In this context, it is the person who posts the

:  We understand and appreciate your objections to Juniper.  Juniper is
: not being rewarded for anything.  Rather, they are being brought online
: as a new CNA so that we can expand the CVE capability consistent with
: the stated objective of our Board colleagues to scale the capability
: under a federated approach to increase coverage.  We were delighted to

So to sum this up:

MITRE made a unilateral decision to make Juniper a CNA, six days after a
board member expressed concerns over their handling of CVE assignments,
and gave board membrs an opportunity to bring up concerns without stating
taht concerns had already been brought up, and that Juniper already had a
history of not following CNA guidelines. That the board members could
bring up concerns in private, with no indication or direction they could
also share the concerns publicly.

Again, remind us what the purpose of the board is exactly, if we're not
directing decisions. More importantly, when we do give input, even
proactively, it is apparently not considered nor brought up when
announcing MITRE's decisions that are made without any board input
whatsoever. I ask because the purpose of the board as seen by the public,
the board members, and MITRE seem to be at odds. Clearing this up would be
helpful for everyone involved.
Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

Kurt Seifried
Actually on the board call this was brought up and I actually brought up your concerns, we all agreed that they weren't blockers and that also with documentation/oversight/feedback this is most likely a manageable problem. We were then asked if anyone still objected strongly or if we were ok with Juniper being made a CNA and it was unanimous in favor. 

Which does bring up one note, voting on phone calls is probably sub optimal due to timezones/etc, we should probably use the private list so that everyone has time to reply and it's recorded more easily if someone does bring up specific concerns/etc.  

On Thu, Apr 21, 2016 at 11:04 PM, jericho <[hidden email]> wrote:
On Wed, 20 Apr 2016, Common Vulnerabilities & Exposures wrote:

: Brian -
:
: to their own opinions, all opinions must be considered.  For example,
: the note to the private Board list yesterday regarding Juniper was
: intended to provide all Board members with an opportunity to privately
: voice opinions in a candid fashion that they may have been uncomfortable
: voicing in public.  In this context, it is the person who posts the

:  We understand and appreciate your objections to Juniper.  Juniper is
: not being rewarded for anything.  Rather, they are being brought online
: as a new CNA so that we can expand the CVE capability consistent with
: the stated objective of our Board colleagues to scale the capability
: under a federated approach to increase coverage.  We were delighted to

So to sum this up:

MITRE made a unilateral decision to make Juniper a CNA, six days after a
board member expressed concerns over their handling of CVE assignments,
and gave board membrs an opportunity to bring up concerns without stating
taht concerns had already been brought up, and that Juniper already had a
history of not following CNA guidelines. That the board members could
bring up concerns in private, with no indication or direction they could
also share the concerns publicly.

Again, remind us what the purpose of the board is exactly, if we're not
directing decisions. More importantly, when we do give input, even
proactively, it is apparently not considered nor brought up when
announcing MITRE's decisions that are made without any board input
whatsoever. I ask because the purpose of the board as seen by the public,
the board members, and MITRE seem to be at odds. Clearing this up would be
helpful for everyone involved.



--

--
Kurt Seifried -- Red Hat -- Product Security -- Cloud
PGP A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
Red Hat Product Security contact: [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Juniper to be added to the official list of CNAs

Common Vulnerabilities & Exposures
In reply to this post by jericho
The CVE Team discussed the valid concerns raised by you during yesterday's Board call and held the announcement of Juniper becoming a CNA until we had the opportunity to have this discussion with our Board colleagues.  While the members agreed that mistakes are made with Juniper and other CNAs, it was the opinion of the Board that bringing on Juniper serves the needs of the community better than by not doing so.  The Board was specifically asked if they objected to bringing Juniper on as a CNA.  No members on the call objected.

The Board is a critical advisory function.  As a Board member, you raised concerns about Juniper.  The CVE Team listened to those concerns and raised them with the Board at our first opportunity to do so where a discussion could be held and we could efficiently work through the discussion.  Per Kurt Seifried's good suggestion this morning, we're happy to move to the private Board list to poll the Board on decisions like these in the future.

Thank you for bringing up your concerns.  We appreciate it.

Regards,

The CVE Team

-----Original Message-----
From: jericho [mailto:[hidden email]]
Sent: Friday, April 22, 2016 1:05 AM
To: Common Vulnerabilities & Exposures <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: RE: Juniper to be added to the official list of CNAs
Importance: High


On Wed, 20 Apr 2016, Common Vulnerabilities & Exposures wrote:

: Brian -
:
: to their own opinions, all opinions must be considered.  For example,
: the note to the private Board list yesterday regarding Juniper was
: intended to provide all Board members with an opportunity to privately
: voice opinions in a candid fashion that they may have been uncomfortable
: voicing in public.  In this context, it is the person who posts the

:  We understand and appreciate your objections to Juniper.  Juniper is
: not being rewarded for anything.  Rather, they are being brought online
: as a new CNA so that we can expand the CVE capability consistent with
: the stated objective of our Board colleagues to scale the capability
: under a federated approach to increase coverage.  We were delighted to

So to sum this up:

MITRE made a unilateral decision to make Juniper a CNA, six days after a board member expressed concerns over their handling of CVE assignments, and gave board membrs an opportunity to bring up concerns without stating taht concerns had already been brought up, and that Juniper already had a history of not following CNA guidelines. That the board members could bring up concerns in private, with no indication or direction they could also share the concerns publicly.

Again, remind us what the purpose of the board is exactly, if we're not directing decisions. More importantly, when we do give input, even proactively, it is apparently not considered nor brought up when announcing MITRE's decisions that are made without any board input whatsoever. I ask because the purpose of the board as seen by the public, the board members, and MITRE seem to be at odds. Clearing this up would be helpful for everyone involved.
Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

jericho
In reply to this post by Kurt Seifried
On Fri, 22 Apr 2016, Kurt Seifried wrote:

: Which does bring up one note, voting on phone calls is probably sub
: optimal due to timezones/etc, we should probably use the private list so
: that everyone has time to reply and it's recorded more easily if someone
: does bring up specific concerns/etc.

Absolutely. The calls should be to discuss ideas in real-time to speed up
the discussions, but any vote at all should be put to the list.
Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

Carsten Eiram
In reply to this post by Common Vulnerabilities & Exposures
On Fri, Apr 22, 2016 at 4:22 PM, Common Vulnerabilities & Exposures <[hidden email]> wrote:

Per Kurt Seifried's good suggestion this morning, we're happy to move to the private Board list to poll the Board on decisions like these in the future.

When initially reading about it, I planned to send an email today criticizing the decision to do votes on board calls. I'm, therefore, glad to now read that MITRE already recognized that voting on the calls is sub optimal and is refraining from this going forward.

It is unlikely that all board members will be present on a call for various reasons. This makes voting on the private list more appropriate to give everyone time to vote and provide feedback. As Kurt also mentions, timezones are another factor. Some of us are in quite different timezones (myself is GMT+1), which makes it more challenging to attend board calls. Voting on calls would, therefore, make it much harder for current and future board members in EMEA and APAC to have any influence on decisions made.

/Carsten

 
Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

Landfield, Kent B

Just to be clear.... Voting on CNAs has not occurred in the past. Or at least not that I can remember. I see no reason to start now.

I agree official votes should be on the list for items we have previously agreed to vote on but rough consensus on board calls is more than enough for most other items.

I personally would not want to start voting on everything  as that would just slow the effort down greatly at a time when rapid improvements are needed.

If geography is an issue for board members then I suggest we understand the locations members are in so we can look at potentially rescheduling the calls to make it possible for all to have a better opportunity to participate.

Kent Landfield
Intel Corporation
[hidden email]<mailto:[hidden email]>
+1.817.637.8026

On Apr 23, 2016, at 1:56 AM, Carsten Eiram <[hidden email]<mailto:[hidden email]>> wrote:

On Fri, Apr 22, 2016 at 4:22 PM, Common Vulnerabilities & Exposures <[hidden email]<mailto:[hidden email]>> wrote:

Per Kurt Seifried's good suggestion this morning, we're happy to move to the private Board list to poll the Board on decisions like these in the future.

When initially reading about it, I planned to send an email today criticizing the decision to do votes on board calls. I'm, therefore, glad to now read that MITRE already recognized that voting on the calls is sub optimal and is refraining from this going forward.

It is unlikely that all board members will be present on a call for various reasons. This makes voting on the private list more appropriate to give everyone time to vote and provide feedback. As Kurt also mentions, timezones are another factor. Some of us are in quite different timezones (myself is GMT+1), which makes it more challenging to attend board calls. Voting on calls would, therefore, make it much harder for current and future board members in EMEA and APAC to have any influence on decisions made.

/Carsten


Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

jericho
On Sat, 23 Apr 2016, Landfield, Kent B wrote:

: Just to be clear.... Voting on CNAs has not occurred in the past. Or at
: least not that I can remember. I see no reason to start now.

Yet, the board used to vote on every single CVE ID assignment. Things
change.

My primary concern is that a CNA who is not following assignment
guidelines ends up causing confusion and headache for those who monitor
their advisories. We've had users and customers mail us asking about CNA
vendor assignment screwups in the past, so it isn't just us noticing. For
the last month, I have steadily increased the number of mails I am sending
to vendors and researchers about CVE assignment problems, sometimes
sending as many as five a day.

If we can better head off that problem, and make sure a potential CNA is
truly ready to step in as one, we should. I don't get the feeling that
most of the board monitors some of these vendors to the degree I do, so I
don't want a rubberstamping discussion via phone to be the only thing
stopping them from getting approved.

: I agree official votes should be on the list for items we have
: previously agreed to vote on but rough consensus on board calls is more
: than enough for most other items.

Everyone appears to agree on this so far, which I am happy to see.

: I personally would not want to start voting on everything as that would
: just slow the effort down greatly at a time when rapid improvements are
: needed.

No, but we also don't want the typical knee-jerk reaction the U.S.
government is well-known for either (and MITRE demonstrated with that
federated ID scheme change nonsense that wasn't discussed with the board).
Taking an extra few days or even weeks to ensure a solution is appropriate
benefits us more than rushing to a solution that will demand more fixes in
the months to come.
Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

Kurt Seifried
Hopefully documentation/training will reduce the screw ups/accidents, and tighter feedback loops will reduce the time to correct. One thing that was also mentioned on the board call was having a public "inventory" of CNAs, how to contact them, etc and also a private "inventory" of contact details/etc (e.g. direct managers). 

On Sat, Apr 23, 2016 at 10:02 AM, jericho <[hidden email]> wrote:
On Sat, 23 Apr 2016, Landfield, Kent B wrote:

: Just to be clear.... Voting on CNAs has not occurred in the past. Or at
: least not that I can remember. I see no reason to start now.

Yet, the board used to vote on every single CVE ID assignment. Things
change.

My primary concern is that a CNA who is not following assignment
guidelines ends up causing confusion and headache for those who monitor
their advisories. We've had users and customers mail us asking about CNA
vendor assignment screwups in the past, so it isn't just us noticing. For
the last month, I have steadily increased the number of mails I am sending
to vendors and researchers about CVE assignment problems, sometimes
sending as many as five a day.

If we can better head off that problem, and make sure a potential CNA is
truly ready to step in as one, we should. I don't get the feeling that
most of the board monitors some of these vendors to the degree I do, so I
don't want a rubberstamping discussion via phone to be the only thing
stopping them from getting approved.

: I agree official votes should be on the list for items we have
: previously agreed to vote on but rough consensus on board calls is more
: than enough for most other items.

Everyone appears to agree on this so far, which I am happy to see.

: I personally would not want to start voting on everything as that would
: just slow the effort down greatly at a time when rapid improvements are
: needed.

No, but we also don't want the typical knee-jerk reaction the U.S.
government is well-known for either (and MITRE demonstrated with that
federated ID scheme change nonsense that wasn't discussed with the board).
Taking an extra few days or even weeks to ensure a solution is appropriate
benefits us more than rushing to a solution that will demand more fixes in
the months to come.



--

--
Kurt Seifried -- Red Hat -- Product Security -- Cloud
PGP A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
Red Hat Product Security contact: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

Art Manion
In reply to this post by Carsten Eiram
On 2016-04-23 02:56, Carsten Eiram wrote:

>     Per Kurt Seifried's good suggestion this morning, we're happy to
>     move to the private Board list to poll the Board on decisions like
>     these in the future.

> When initially reading about it, I planned to send an email today
> criticizing the decision to do votes on board calls. I'm, therefore,
> glad to now read that MITRE already recognized that voting on the calls
> is sub optimal and is refraining from this going forward.

Yes, if votes count, we need a written ballot, voting deadlines, and
electronic/written tallies.  Or if votes happen on a call, voters need
the ballot in advance.  Might need more rules like quorum and proxy.

I do believe that the board is editorial/advisory only.  For example, if
the board votes 9-1 against a new CNA, can MITRE just approve the CNA
anyway?  I'm not suggesting MITRE would just ignore board input
(especially a 9-1 opinion), but before we go to the effort of setting up
voting rules, I'd like to be clear what power board votes actually have.

 - Art
Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

Carsten Eiram
On Mon, Apr 25, 2016 at 5:28 AM, Art Manion <[hidden email]> wrote:
On 2016-04-23 02:56, Carsten Eiram wrote:

I do believe that the board is editorial/advisory only.  For example, if
the board votes 9-1 against a new CNA, can MITRE just approve the CNA
anyway?  I'm not suggesting MITRE would just ignore board input
(especially a 9-1 opinion), but before we go to the effort of setting up
voting rules, I'd like to be clear what power board votes actually have.


Yes, it is my understanding too that the Board is advisory only. That is not a concern of mine; MITRE should have the freedom to go against a vote for whatever reasons they may have. I would just in such cases expect MITRE to provide their reasoning for disregarding the vote.

Even so, one must presume that each of us were elected on the Board for a reason and based on our varying backgrounds may be able to contribute with insight that others do not necessarily possess. It, therefore, seems in MITRE's best interest to ensure they get feedback and votes from all Board members during anything that is deemed important enough to vote about. On that note, I actually care more about the reasoning provided by each Board member when voting than the vote itself. If one Board member clearly has more knowledge / experience about a given issue being voted on, I'd consider that feedback (and resulting vote) more valuable than just looking at the majority vote alone. Hopefully, MITRE does the same.

Finally, MITRE recently stated that they considered the charters "dated and in need of updating to better reflect the Board's advisory role". It seems appropriate that MITRE prioritizes updating the charter to have it reflect their current perception of the Board's role. That'll allow us to better understand what is expected of Board members as well as the power of votes and feedback.


Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

Landfield, Kent B
In reply to this post by Landfield, Kent B
Carsten wrote:
"Some of us are in quite different timezones (myself is GMT+1), which makes it more challenging to attend board calls. Voting on calls would, therefore, make it much harder for current and future board members in EMEA and APAC to have any influence on decisions made."

Currently the next Board call will occur on May 5th at 2:00 PM until 4:00 PM EDT.  That would be a six hour difference for you so it would be  8:00 PM to 10:00 PM.  While I know this is later than normal working days. Will this be an on-going problem for you to be able to participate in?

I am not sure we have any members in APAC.  As I mentioned, "If geography is an issue for board members then I suggest we understand the locations members are in so we can look at potentially rescheduling the calls to make it possible for all to have a better opportunity to participate.”

Are there others outside the US that have an issue with the time the recurring meeting is scheduled?

Thanks.
---
Kent Landfield
+1.817.637.8026

Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

Carsten Eiram
In reply to this post by Art Manion
On Mon, Apr 25, 2016 at 6:15 AM, Kurt Seifried <[hidden email]> wrote:
I would suggest it's less about the voting per se and more about the
concerns raised,

I find it's about both. However, the main concern regarding the voting was not as much about the Juniper vote itself, as it was about the precedence it could set for votes in the future generally being on Board calls vs. the private list that has historically been used quite successfully.


e.g. With juniper some legitimate issues were raised

Indeed some very legitimate issues.


but by and large everyone on the call (at least my impression) was
like "it's not 100% perfect. It it's not a show stopper, and chances
are the problems can be rectified without to much hassle." As
evidenced if a good chunk of the board (or a unanimous majority
attending) then does vote in the affirmative I think it's safe to say
it's ok to proceed with no material concerns.

The flaw in that logic is that it is assumed all of us roughly have the same knowledge about a given issue, and that the majority is the smartest. The experts rarely comprise the majority. Each of us have different backgrounds and are not equally clever in all areas.

Had Brian not voiced his concerns, I bet no-one else on the Board would even know Juniper today are struggling when assigning CVE identifiers. Although Brian did have a chance to voice his concerns and provide some details, I doubt anyone still really understand the extent except him (and possibly MITRE depending on how many details he provided them off-list). Naturally, it's the responsibility of the Board member voicing a concern to provide sufficient details to allow the rest of us to make an informed decision, but in this case it seems no real attempts were made to further explore how bad it was before voting.

As someone who didn't attend the call and just have the list discussions to form an opinion on, it comes across as if Brian's concerns were more or less dismissed, and that MITRE and Board members were more eager to get another CNA onboard vs. taking the time to fully explore the concerns raised. Why the rush? Because it's important for MITRE to show that they're making progress on the CNA front, which is clear from their initial email announcement.

If a Board member with significant insight into the CVE assignment practises of a given vendor voices concern, it seems prudent to fully investigate said concerns and at minimum raise those with the vendor and get a commitment from said vendor to address those _before_ allowing them to become a CNA. Instead I experience what I can best describe as a laissez-faire attitude: We'll just make them a CNA since no-one (except Brian Martin, who wasn't on the call) "objected strongly", and any issues are likely a "manageable problem" that hopefully documentation/training can fix at some point in the future.

What are the plans now to follow up with Juniper to get said issues rectified?

We do need more CNAs, but if they don't understand the basics of CVE assignments, it may be more hassle than value. We already see plenty of CNAs not following CVE abstraction guidelines without MITRE doing anything about it. Alternatively, we need to define the level of CVE compliance we actually care about. Is it more important to have a lot of CNAs assigning CVEs to issues vs. following the CVE abstraction guidelines, which then should just be treated as rough pointers? That may be a matter for the Board to discuss. Personally, I find that the abstraction guidelines are there for a reason, and it is fair to at minimum expect a CNA to be fully familiar and comply with them.


I would assume had like
half the board voted against mitre might take a second look at things
before proceeding

I'm not so sure. The decision to make Juniper a CNA seemed pretty much done from the initial announcement, which just stated that Juniper had met all requirements and that a public announcement would go out the same afternoon. The follow-up mail from MITRE stated that the heads-up on the private list was provided "as a courtesy". While it did include that the reason for said courtesy was to allow Board members to raise "significant issues" with naming Juniper a CNA, the Board was only given a few hours, so MITRE clearly didn't expect much push-back if any. Furthermore, nowhere did the first email notification suggest that MITRE was actually looking for Board feedback.

I have no issue with MITRE deciding who becomes CNAs without the Board voting. Either approach works for me, and I'm certainly not advocating that we need to vote on everything.

I do, however, think the Board needs to know what the CNA requirements are. It's problematic when MITRE just states that a given vendor "has met all of the requirements" when it's unclear to us what these are. More so when it's disputed by a Board member. I worked with MITRE when Secunia became a CNA many years ago (they lost that status again after I left), so I know what requirements were asked of me/us back then. Mr. (ex-)CVE aka Steve Christey put me through the wringer. Based on Brian's feedback, Juniper certainly haven't met all those requirements if unchanged.

Lastly, I understand that Brian informed MITRE about some assignment problems with Juniper prior to the call. That makes it even more important for us to understand what the current CNA guidelines are, how MITRE is ensuring that they're being followed, and that such assignment concerns are addressed before a new CNA is created. Otherwise, it could lead to a long list of problems in the future. Did MITRE discuss Juniper's potential CNA status when the concerns were voiced by Brian? I know Kurt mentioned Brian's concerns on the call, but did MITRE also specifically bring up the concerns in a detailed manner to ensure Board members not as familiar with Juniper's CVE assignments could factor that in?


(much like with the DWF proposal, if I can't
convince most people  it's a good idea, then there's a good chance it
isn't a good idea).

While I follow what you're saying, I, again, don't subscribe to "majority knows best". I'd rather listen to a few people, who know what they're talking about vs. getting uninformed thoughts from the majority. Especially as a community-driven project, the majority of people have various agendas and may express love for the idea without having any intentions of contributing or insights in order for it to become successful.


Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

Carsten Eiram
In reply to this post by Landfield, Kent B
On Mon, Apr 25, 2016 at 5:29 PM, Landfield, Kent B <[hidden email]> wrote:
Carsten wrote:
"Some of us are in quite different timezones (myself is GMT+1), which makes it more challenging to attend board calls. Voting on calls would, therefore, make it much harder for current and future board members in EMEA and APAC to have any influence on decisions made."

Currently the next Board call will occur on May 5th at 2:00 PM until 4:00 PM EDT.  That would be a six hour difference for you so it would be  8:00 PM to 10:00 PM.  While I know this is later than normal working days. Will this be an on-going problem for you to be able to participate in?

For the most part, yes. However, we naturally have the Board calls when most members are able to participate, and I see no problem with that. Do the Board calls to facilitate easier discussion of issues, but reserve voting for the private list after meeting minutes have been posted to allow all Board members to provide input and vote.


 
Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

Art Manion
In reply to this post by Carsten Eiram
On 2016-04-26 03:01, Carsten Eiram wrote:

...
> As someone who didn't attend the call and just have the list discussions
> to form an opinion on, it comes across as if Brian's concerns were more
> or less dismissed, and that MITRE and Board members were more eager to
> get another CNA onboard vs. taking the time to fully explore the
> concerns raised. Why the rush? Because it's important for MITRE to show
> that they're making progress on the CNA front, which is clear from their
> initial email announcement.

I noted Brian's concerns and do not doubt that they have merit.  I also
generally agree with your assessment of the situation.  But, to me, it's
simply more important to expand CVE, and part of that means more CNAs,
particularly vendor CNAs who should be largely responsible for
assignments in their own software.

I believe Brian that Juniper has issues.  I have first hand experience
with another vendor CNA who has not followed the rules.  I'm pretty sure
there are other examples.

Without refreshed CNA governance rules, it doesn't matter a whole lot.
Once the rules are in place and being reasonably enforced, Juniper can
follow them or face the consequences, like all the CNAs.

Speaking of consequences, what if Juniper doesn't follow the rules?
Withdraw their CNA status?  Then who is going to issue CVE IDs for
Juniper vulnerabilities?  If a CNA assigns incorrectly, reject their
assignments.  If the CNA actually wants their CVE IDs to count, they'll
shape up.  If they don't, de-list them.  And yes, this does sound like
laissez-faire.  The current model doesn't scale.

Growing CVE is going decrease fidelity.  As far as I've thought about
it, MITRE acting as CNA registrar/auditor/manager and ultimate arbiter
of assignments from many CNAs might work as an organizational model.

 - Art
Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

jericho
On Wed, 27 Apr 2016, Art Manion wrote:

: I believe Brian that Juniper has issues.  I have first hand experience
: with another vendor CNA who has not followed the rules.  I'm pretty sure
: there are other examples.

A bit of an understatement. =)

Almost every single CNA has had screw-ups in assignments in the last 12
months, including Oracle, Microsoft, and Adobe. The one CNA I can't find
fault with lately is Silicon Graphics.

: Speaking of consequences, what if Juniper doesn't follow the rules?
: Withdraw their CNA status?  Then who is going to issue CVE IDs for
: Juniper vulnerabilities?  If a CNA assigns incorrectly, reject their
: assignments.  If the CNA actually wants their CVE IDs to count, they'll
: shape up.  If they don't, de-list them.  And yes, this does sound like
: laissez-faire.  The current model doesn't scale.

And I have spoken to this point as well. We don't just need rules, we need
a clear path on how MITRE will deal with them if they aren't following
rules. Unless MITRE decided to keep me out of the loop after I reported
CNAs not following rules many times, then I don't believe MITRE has been
following up with them much at all. Or perhaps for a fraction of my
complaints.

I can't imagine MITRE will actually revoke a CNA, because it goes against
their selfish interests (CVE is part of a multi-million dollar contract
they enjoy every year). That is a grim reality we need to remember as we
discuss this problem. I only bring it up because many of us had proposed
that MITRE bring on more CNAs several years ago, and that was met with
silence or opposition (usually in private). Now that they are being called
to task, it seems greenlighting new CNAs could be their answer, even if
the vendor has a history of bad assignments and board members object.

I think what bothers me about this discussion isn't just that I had issues
with Juniper before the CNA status came up, but now that it is public...
what is happening? It would take less than eight hours for one of the
abundant MITRE employees tasked with CVE duties to audit Juniper's
advisories for the last couple of years, and determine how accurate their
assignments are. Given that Juniper has been requesting those IDs from
MITRE, they could further compare the email requests to the public
advisories to really gauge Juniper's understanding of the process. That is
something I cannot do, since I don't see the ID request emails. Yet, I
track CNA failures in a passing degree via several other data aggregation
initiatives that have a side effect of giving me that data, and more.

Eight hours of figuring out where Juniper stands in this process is a
no-brainer to me, given that every bad public assignment can snowball and
cause serious grief for their customers, and in turn for any CVE customer.
The ROI on such a brief audit is clear.

In fact, every CNA, current or proposed, should be audited once a year, to
ensure they are following assignment guidelines. What seems minor and
pedestrian on the surface to many (e.g. assigning a 2016 ID to a 2015
issue), can also snowball in huge ways, as seen in the 2016 Verizon DBIR
report (pg13, 'Vulnerabilities' section) where the methodology is not
defined, and they may be using the year of the ID to attribute disclosure
attributes. Even if they don't, *many* others have historically done just
that when generating yearly vuln totals based on CVE data. These stats are
about the only you see in any media, industry or mainstream. Because CVE
didn't think that 'disclosure date' was important to track in 1999, means
almost every vulnerability stat today is absurd and wrong.

: Growing CVE is going decrease fidelity.  As far as I've thought about
: it, MITRE acting as CNA registrar/auditor/manager and ultimate arbiter
: of assignments from many CNAs might work as an organizational model.

In a perfect CVE world, MITRE would only act as a manager and auditor of
CNAs and do no assignments themselves (I could also argue they aren't as
qualified to do so anymore, but that is academic pedantry and a losing
argument due to social perception, not fact). I don't get how it is 2016
and this is just being brought up as a possible model falls somewhere
between amusing and disgusting, especially since I have never seen MITRE
propose it, while half a dozen other industry professionals have in the
previous years.
Reply | Threaded
Open this post in threaded view
|

Re: Juniper to be added to the official list of CNAs

Kurt Seifried


On Tue, Apr 26, 2016 at 11:28 PM, jericho <[hidden email]> wrote:
On Wed, 27 Apr 2016, Art Manion wrote:


: Speaking of consequences, what if Juniper doesn't follow the rules?
: Withdraw their CNA status?  Then who is going to issue CVE IDs for
: Juniper vulnerabilities?  If a CNA assigns incorrectly, reject their
: assignments.  If the CNA actually wants their CVE IDs to count, they'll
: shape up.  If they don't, de-list them.  And yes, this does sound like
: laissez-faire.  The current model doesn't scale.

And I have spoken to this point as well. We don't just need rules, we need
a clear path on how MITRE will deal with them if they aren't following
rules. Unless MITRE decided to keep me out of the loop after I reported
CNAs not following rules many times, then I don't believe MITRE has been
following up with them much at all. Or perhaps for a fraction of my
complaints.

I can't imagine MITRE will actually revoke a CNA, because it goes against
their selfish interests (CVE is part of a multi-million dollar contract
they enjoy every year). That is a grim reality we need to remember as we
discuss this problem. I only bring it up because many of us had proposed
that MITRE bring on more CNAs several years ago, and that was met with
silence or opposition (usually in private). Now that they are being called
to task, it seems greenlighting new CNAs could be their answer, even if
the vendor has a history of bad assignments and board members object.

I would rather see education/etc rather than revocation (classic rehabilitate vs punish). Revoking a CNA also can result in "damage", assuming the former CNA still does CVE they would take longer (I assume), or they stop doing CVE entirely. 

My plan with DWF is to think about it and ideally avoid the problem, but if it occurs to start with education and if that doesn't work then we ramp up from there.
 
In fact, every CNA, current or proposed, should be audited once a year, to
ensure they are following assignment guidelines. What seems minor and
pedestrian on the surface to many (e.g. assigning a 2016 ID to a 2015
issue), can also snowball in huge ways, as seen in the 2016 Verizon DBIR
report (pg13, 'Vulnerabilities' section) where the methodology is not
defined, and they may be using the year of the ID to attribute disclosure
attributes. Even if they don't, *many* others have historically done just
that when generating yearly vuln totals based on CVE data. These stats are
about the only you see in any media, industry or mainstream. Because CVE
didn't think that 'disclosure date' was important to track in 1999, means
almost every vulnerability stat today is absurd and wrong.

The audit would be problematic for vendors with a lot of CVEs (Red Hat) or for vendors with limited information disclosure (no names but we all know at least one...). I'm more interested in ensuring that CVE process and CNAs/Mitre provide feedback loops so we can identify and fix these problems. Maybe such examples should be posted to the list (assuming we don't flood it)? 

--
Kurt Seifried -- Red Hat -- Product Security -- Cloud
PGP A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
Red Hat Product Security contact: [hidden email]