Current standards/criteria for 'Undefined Behavior'

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Current standards/criteria for 'Undefined Behavior'

jericho
MITRE,

Can you outline the current standards, criteria, or guidelines you use for
assigning an ID to an issue that is simply 'undefined behavior' with no
indication of exploitability or crossing privilege boundaries? We're
seeing these a bit more frequently lately and they often appear to get an
ID without any examination by the researcher or MITRE. In many cases,
subsequent analysis determines these are non-issues and are not
exploitable.

Thanks,

Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Current standards/criteria for 'Undefined Behavior'

Evans, Jonathan L.
Brian,

We are not entirely sure what you mean by "undefined behavior."  Are you talking about the CVE entries that specifically say "undefined behavior," such as CVE-2017-8326 and CVE-2017-7961?

Thanks,
Jonathan

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of jericho
Sent: Tuesday, May 02, 2017 1:26 PM
To: cve-editorial-board-list <[hidden email]>
Subject: Current standards/criteria for 'Undefined Behavior'
Importance: High

MITRE,

Can you outline the current standards, criteria, or guidelines you use for assigning an ID to an issue that is simply 'undefined behavior' with no indication of exploitability or crossing privilege boundaries? We're seeing these a bit more frequently lately and they often appear to get an ID without any examination by the researcher or MITRE. In many cases, subsequent analysis determines these are non-issues and are not exploitable.

Thanks,

Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Current standards/criteria for 'Undefined Behavior'

jericho
Jonathan,

On Wed, 3 May 2017, Evans, Jonathan L. wrote:

: We are not entirely sure what you mean by "undefined behavior."  Are you
: talking about the CVE entries that specifically say "undefined
: behavior," such as CVE-2017-8326 and CVE-2017-7961?

Yes. A short list:

https://www.google.com/search?q=site%3Acve.mitre.org+%22undefined+behavior%22&oq=site%3Acve.mitre.org+%22undefined+behavior%22&aqs=chrome..69i57j69i58.3867j0j7&sourceid=chrome&ie=UTF-8

Thank you,

Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Current standards/criteria for 'Undefined Behavior'

Adinolfi, Daniel R

All,

 

The CVE Team largely agrees with community discussion of this topic at:

  http://openwall.com/lists/oss-security/2017/04/12/3

  http://openwall.com/lists/oss-security/2017/04/12/2

In terms of the CNA Rules, we feel that these undefined-behavior reports satisfy the definition of a vulnerability (Appendix A) because it is unreasonable for computational logic to depend on the implementation details of a compiler or the selected compiler options.

All, or nearly all, of the recent "undefined behavior" CVE IDs are the result of a discloser relying on the UndefinedBehaviorSanitizer tool that has design goals expressed in the http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html post. Also, these CVE IDs are for software products that are libraries, usable in building arbitrary programs including long-running daemons.

Referring to the C standard at http://www.open-std.org/jtc1/sc22/wg14/www/standards.html or similar online versions such as the http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf document, one can see statements such as "Possible undefined behavior ranges from ignoring the situation completely with unpredictable results ... to terminating a translation or execution." Terminating an execution is probably the most relevant for these CVE IDs. Also, the document

specifically mentions types of undefined behavior for which CVE IDs have been assigned, e.g., "If the value being converted is outside the range of values that can be represented, the behavior is undefined" and "The result of E1 << E2 is E1 left-shifted E2 bit positions ... otherwise, the behavior is undefined."

In other words, a program termination (aka a crash) is an outcome called out in the C standard. A crash within library code has a loss of availability, and thus matches the definition of a vulnerability.

 

One scenario of interest is a user who explicitly chooses compiler options that terminate the program upon encountering undefined behavior, e.g., "clang -fsanitize=undefined -fno-sanitize-recover" with LLVM. This means that an availability impact does occur. Admittedly, the availability impact may be rare in the real world, because people don't often use those clang flags for production code.

Another scenario of interest is compiler optimizations that happen to result in generation of vulnerable code for a program that relies on undefined behavior, such as CVE-2017-7892 which is about undefined behavior in "pointer arithmetic calculations that overflow." Here, the impact might not be limited to availability.

Based on the above, the MITRE CNA currently uses this guideline: a report of undefined behavior in a library is considered equivalent to a report of a crash in a library, in determining whether a CVE ID may exist.

However, Appendix E of the CNA Rules is also followed. For example, in the CVE-2017-7961 case, we received a dispute stating that there is no impact. We recorded the dispute as required (this does not necessarily mean that we believe the dispute is correct).

 

Please let us know if you have any other questions or need any other clarification.

 

Thanks.

 

-Dan, for the Team

 

From: <[hidden email]> on behalf of jericho <[hidden email]>
Date: Monday, May 8, 2017 at 19:50
To: "Evans, Jonathan L." <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: RE: Current standards/criteria for 'Undefined Behavior'

 

Jonathan,

 

On Wed, 3 May 2017, Evans, Jonathan L. wrote:

 

: We are not entirely sure what you mean by "undefined behavior."  Are you

: talking about the CVE entries that specifically say "undefined

: behavior," such as CVE-2017-8326 and CVE-2017-7961?

 

Yes. A short list:

 

<a href="https://www.google.com/search?q=site%3Acve.mitre.org&#43;%22undefined&#43;behavior%22&amp;oq=site%3Acve.mitre.org&#43;%22undefined&#43;behavior%22&amp;aqs=chrome..69i57j69i58.3867j0j7&amp;sourceid=chrome&amp;ie=UTF-8">https://www.google.com/search?q=site%3Acve.mitre.org+%22undefined+behavior%22&oq=site%3Acve.mitre.org+%22undefined+behavior%22&aqs=chrome..69i57j69i58.3867j0j7&sourceid=chrome&ie=UTF-8

 

Thank you,

 

Brian

 

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Current standards/criteria for 'Undefined Behavior'

Carsten Eiram
I hope the new MITRE CVE team realizes they are in a minority of people in this industry, who actually consider such issues as being CVE worthy by default or even security-relevant without some proof of there being a (realistic) security impact.

Referencing Hanno Böck and Agostino Sarubbo of all people does not lend a lot of "street cred" or underline the point. These guys are fairly inexperienced when it comes to vulnerability research and focus on fuzzing with AFL. One of them (Hanno) until recently didn't even really understand the output; the other still struggles.

We do not disagree that issues leading to undefined behaviour _theoretically_ have a security impact. Rarely is it ever proven, though. In fact, I don't think Agostino Sarubbo (or Hanno for that matter) has proven a single of the UBSan issues, which he has reported many of, actually did have a real-world impact.

Most experienced people in this industry have historically expected some reasonable indication of a security impact that was not too theoretical. There is a reason for us not seeing these types of reports from experienced researchers, who are also fuzzing using AFL. Even the OSS-Fuzz project flags UBSan issues as "Bug" and not "Bug-Security" by default. Sure, they should be fixed for good measure and are bad coding practice, but even developers generally don't treat the fixes as security-relevant. And rightly so.

MITRE is humoring beginners, who are fuzzing and just dumping their UBSan output without doing minimal due diligence; it would be better to educate them to higher standards i.e. the standards of most vulnerability researchers as well as the old MITRE CVE team. Otherwise, I hope the new MITRE CVE team is wearing their track pants and are on top of their cardio. You'll need some excellent stamina when taking on the CVE assignments for the metric ton of UBSan issues you'll be dealing with, if that is your default position.

/Carsten


On Wed, May 10, 2017 at 4:04 PM, Adinolfi, Daniel R <[hidden email]> wrote:

All,

 

The CVE Team largely agrees with community discussion of this topic at:

  http://openwall.com/lists/oss-security/2017/04/12/3

  http://openwall.com/lists/oss-security/2017/04/12/2

In terms of the CNA Rules, we feel that these undefined-behavior reports satisfy the definition of a vulnerability (Appendix A) because it is unreasonable for computational logic to depend on the implementation details of a compiler or the selected compiler options.

All, or nearly all, of the recent "undefined behavior" CVE IDs are the result of a discloser relying on the UndefinedBehaviorSanitizer tool that has design goals expressed in the http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html post. Also, these CVE IDs are for software products that are libraries, usable in building arbitrary programs including long-running daemons.

Referring to the C standard at http://www.open-std.org/jtc1/sc22/wg14/www/standards.html or similar online versions such as the http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf document, one can see statements such as "Possible undefined behavior ranges from ignoring the situation completely with unpredictable results ... to terminating a translation or execution." Terminating an execution is probably the most relevant for these CVE IDs. Also, the document

specifically mentions types of undefined behavior for which CVE IDs have been assigned, e.g., "If the value being converted is outside the range of values that can be represented, the behavior is undefined" and "The result of E1 << E2 is E1 left-shifted E2 bit positions ... otherwise, the behavior is undefined."

In other words, a program termination (aka a crash) is an outcome called out in the C standard. A crash within library code has a loss of availability, and thus matches the definition of a vulnerability.

 

One scenario of interest is a user who explicitly chooses compiler options that terminate the program upon encountering undefined behavior, e.g., "clang -fsanitize=undefined -fno-sanitize-recover" with LLVM. This means that an availability impact does occur. Admittedly, the availability impact may be rare in the real world, because people don't often use those clang flags for production code.

Another scenario of interest is compiler optimizations that happen to result in generation of vulnerable code for a program that relies on undefined behavior, such as CVE-2017-7892 which is about undefined behavior in "pointer arithmetic calculations that overflow." Here, the impact might not be limited to availability.

Based on the above, the MITRE CNA currently uses this guideline: a report of undefined behavior in a library is considered equivalent to a report of a crash in a library, in determining whether a CVE ID may exist.

However, Appendix E of the CNA Rules is also followed. For example, in the CVE-2017-7961 case, we received a dispute stating that there is no impact. We recorded the dispute as required (this does not necessarily mean that we believe the dispute is correct).

 

Please let us know if you have any other questions or need any other clarification.

 

Thanks.

 

-Dan, for the Team

 

From: <[hidden email]> on behalf of jericho <[hidden email]>
Date: Monday, May 8, 2017 at 19:50
To: "Evans, Jonathan L." <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: RE: Current standards/criteria for 'Undefined Behavior'

 

Jonathan,

 

On Wed, 3 May 2017, Evans, Jonathan L. wrote:

 

: We are not entirely sure what you mean by "undefined behavior."  Are you

: talking about the CVE entries that specifically say "undefined

: behavior," such as CVE-2017-8326 and CVE-2017-7961?

 

Yes. A short list:

 

 

Thank you,

 

Brian

 


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Current standards/criteria for 'Undefined Behavior'

Millar, Thomas

I tend to agree with Carsten here but I assume a response further explaining the CVE Team’s reasoning is being drafted already.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Carsten Eiram
Sent: 11 May, 2017 07:19
To: Adinolfi, Daniel R <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: Re: Current standards/criteria for 'Undefined Behavior'

 

I hope the new MITRE CVE team realizes they are in a minority of people in this industry, who actually consider such issues as being CVE worthy by default or even security-relevant without some proof of there being a (realistic) security impact.

 

Referencing Hanno Böck and Agostino Sarubbo of all people does not lend a lot of "street cred" or underline the point. These guys are fairly inexperienced when it comes to vulnerability research and focus on fuzzing with AFL. One of them (Hanno) until recently didn't even really understand the output; the other still struggles.

 

We do not disagree that issues leading to undefined behaviour _theoretically_ have a security impact. Rarely is it ever proven, though. In fact, I don't think Agostino Sarubbo (or Hanno for that matter) has proven a single of the UBSan issues, which he has reported many of, actually did have a real-world impact.

 

Most experienced people in this industry have historically expected some reasonable indication of a security impact that was not too theoretical. There is a reason for us not seeing these types of reports from experienced researchers, who are also fuzzing using AFL. Even the OSS-Fuzz project flags UBSan issues as "Bug" and not "Bug-Security" by default. Sure, they should be fixed for good measure and are bad coding practice, but even developers generally don't treat the fixes as security-relevant. And rightly so.

 

MITRE is humoring beginners, who are fuzzing and just dumping their UBSan output without doing minimal due diligence; it would be better to educate them to higher standards i.e. the standards of most vulnerability researchers as well as the old MITRE CVE team. Otherwise, I hope the new MITRE CVE team is wearing their track pants and are on top of their cardio. You'll need some excellent stamina when taking on the CVE assignments for the metric ton of UBSan issues you'll be dealing with, if that is your default position.

 

/Carsten

 

 

On Wed, May 10, 2017 at 4:04 PM, Adinolfi, Daniel R <[hidden email]> wrote:

All,

 

The CVE Team largely agrees with community discussion of this topic at:

  http://openwall.com/lists/oss-security/2017/04/12/3

  http://openwall.com/lists/oss-security/2017/04/12/2

In terms of the CNA Rules, we feel that these undefined-behavior reports satisfy the definition of a vulnerability (Appendix A) because it is unreasonable for computational logic to depend on the implementation details of a compiler or the selected compiler options.

All, or nearly all, of the recent "undefined behavior" CVE IDs are the result of a discloser relying on the UndefinedBehaviorSanitizer tool that has design goals expressed in the http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html post. Also, these CVE IDs are for software products that are libraries, usable in building arbitrary programs including long-running daemons.

Referring to the C standard at http://www.open-std.org/jtc1/sc22/wg14/www/standards.html or similar online versions such as the http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf document, one can see statements such as "Possible undefined behavior ranges from ignoring the situation completely with unpredictable results ... to terminating a translation or execution." Terminating an execution is probably the most relevant for these CVE IDs. Also, the document

specifically mentions types of undefined behavior for which CVE IDs have been assigned, e.g., "If the value being converted is outside the range of values that can be represented, the behavior is undefined" and "The result of E1 << E2 is E1 left-shifted E2 bit positions ... otherwise, the behavior is undefined."

In other words, a program termination (aka a crash) is an outcome called out in the C standard. A crash within library code has a loss of availability, and thus matches the definition of a vulnerability.

 

One scenario of interest is a user who explicitly chooses compiler options that terminate the program upon encountering undefined behavior, e.g., "clang -fsanitize=undefined -fno-sanitize-recover" with LLVM. This means that an availability impact does occur. Admittedly, the availability impact may be rare in the real world, because people don't often use those clang flags for production code.

Another scenario of interest is compiler optimizations that happen to result in generation of vulnerable code for a program that relies on undefined behavior, such as CVE-2017-7892 which is about undefined behavior in "pointer arithmetic calculations that overflow." Here, the impact might not be limited to availability.

Based on the above, the MITRE CNA currently uses this guideline: a report of undefined behavior in a library is considered equivalent to a report of a crash in a library, in determining whether a CVE ID may exist.

However, Appendix E of the CNA Rules is also followed. For example, in the CVE-2017-7961 case, we received a dispute stating that there is no impact. We recorded the dispute as required (this does not necessarily mean that we believe the dispute is correct).

 

Please let us know if you have any other questions or need any other clarification.

 

Thanks.

 

-Dan, for the Team

 

From: <[hidden email]> on behalf of jericho <[hidden email]>
Date: Monday, May 8, 2017 at 19:50
To: "Evans, Jonathan L." <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: RE: Current standards/criteria for 'Undefined Behavior'

 

Jonathan,

 

On Wed, 3 May 2017, Evans, Jonathan L. wrote:

 

: We are not entirely sure what you mean by "undefined behavior."  Are you

: talking about the CVE entries that specifically say "undefined

: behavior," such as CVE-2017-8326 and CVE-2017-7961?

 

Yes. A short list:

 

<a href="https://www.google.com/search?q=site%3Acve.mitre.org&#43;%22undefined&#43;behavior%22&amp;oq=site%3Acve.mitre.org&#43;%22undefined&#43;behavior%22&amp;aqs=chrome..69i57j69i58.3867j0j7&amp;sourceid=chrome&amp;ie=UTF-8" target="_blank">https://www.google.com/search?q=site%3Acve.mitre.org+%22undefined+behavior%22&oq=site%3Acve.mitre.org+%22undefined+behavior%22&aqs=chrome..69i57j69i58.3867j0j7&sourceid=chrome&ie=UTF-8

 

Thank you,

 

Brian

 

 

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Current standards/criteria for 'Undefined Behavior'

Levendis, Chris
We're making it an agenda item for the next Board call.

Chris Levendis


Sent with BlackBerry Work
(www.blackberry.com)

From: Millar, Thomas <[hidden email]>
Date: Thursday, May 11, 2017, 4:45 PM
To: Carsten Eiram <[hidden email]>, Adinolfi, Daniel R <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: RE: Current standards/criteria for 'Undefined Behavior'

I tend to agree with Carsten here but I assume a response further explaining the CVE Team’s reasoning is being drafted already.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Carsten Eiram
Sent: 11 May, 2017 07:19
To: Adinolfi, Daniel R <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: Re: Current standards/criteria for 'Undefined Behavior'

 

I hope the new MITRE CVE team realizes they are in a minority of people in this industry, who actually consider such issues as being CVE worthy by default or even security-relevant without some proof of there being a (realistic) security impact.

 

Referencing Hanno Böck and Agostino Sarubbo of all people does not lend a lot of "street cred" or underline the point. These guys are fairly inexperienced when it comes to vulnerability research and focus on fuzzing with AFL. One of them (Hanno) until recently didn't even really understand the output; the other still struggles.

 

We do not disagree that issues leading to undefined behaviour _theoretically_ have a security impact. Rarely is it ever proven, though. In fact, I don't think Agostino Sarubbo (or Hanno for that matter) has proven a single of the UBSan issues, which he has reported many of, actually did have a real-world impact.

 

Most experienced people in this industry have historically expected some reasonable indication of a security impact that was not too theoretical. There is a reason for us not seeing these types of reports from experienced researchers, who are also fuzzing using AFL. Even the OSS-Fuzz project flags UBSan issues as "Bug" and not "Bug-Security" by default. Sure, they should be fixed for good measure and are bad coding practice, but even developers generally don't treat the fixes as security-relevant. And rightly so.

 

MITRE is humoring beginners, who are fuzzing and just dumping their UBSan output without doing minimal due diligence; it would be better to educate them to higher standards i.e. the standards of most vulnerability researchers as well as the old MITRE CVE team. Otherwise, I hope the new MITRE CVE team is wearing their track pants and are on top of their cardio. You'll need some excellent stamina when taking on the CVE assignments for the metric ton of UBSan issues you'll be dealing with, if that is your default position.

 

/Carsten

 

 

On Wed, May 10, 2017 at 4:04 PM, Adinolfi, Daniel R <[hidden email]> wrote:

All,

 

The CVE Team largely agrees with community discussion of this topic at:

  http://openwall.com/lists/oss-security/2017/04/12/3

  http://openwall.com/lists/oss-security/2017/04/12/2

In terms of the CNA Rules, we feel that these undefined-behavior reports satisfy the definition of a vulnerability (Appendix A) because it is unreasonable for computational logic to depend on the implementation details of a compiler or the selected compiler options.

All, or nearly all, of the recent "undefined behavior" CVE IDs are the result of a discloser relying on the UndefinedBehaviorSanitizer tool that has design goals expressed in the http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html post. Also, these CVE IDs are for software products that are libraries, usable in building arbitrary programs including long-running daemons.

Referring to the C standard at http://www.open-std.org/jtc1/sc22/wg14/www/standards.html or similar online versions such as the http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf document, one can see statements such as "Possible undefined behavior ranges from ignoring the situation completely with unpredictable results ... to terminating a translation or execution." Terminating an execution is probably the most relevant for these CVE IDs. Also, the document

specifically mentions types of undefined behavior for which CVE IDs have been assigned, e.g., "If the value being converted is outside the range of values that can be represented, the behavior is undefined" and "The result of E1 << E2 is E1 left-shifted E2 bit positions ... otherwise, the behavior is undefined."

In other words, a program termination (aka a crash) is an outcome called out in the C standard. A crash within library code has a loss of availability, and thus matches the definition of a vulnerability.

 

One scenario of interest is a user who explicitly chooses compiler options that terminate the program upon encountering undefined behavior, e.g., "clang -fsanitize=undefined -fno-sanitize-recover" with LLVM. This means that an availability impact does occur. Admittedly, the availability impact may be rare in the real world, because people don't often use those clang flags for production code.

Another scenario of interest is compiler optimizations that happen to result in generation of vulnerable code for a program that relies on undefined behavior, such as CVE-2017-7892 which is about undefined behavior in "pointer arithmetic calculations that overflow." Here, the impact might not be limited to availability.

Based on the above, the MITRE CNA currently uses this guideline: a report of undefined behavior in a library is considered equivalent to a report of a crash in a library, in determining whether a CVE ID may exist.

However, Appendix E of the CNA Rules is also followed. For example, in the CVE-2017-7961 case, we received a dispute stating that there is no impact. We recorded the dispute as required (this does not necessarily mean that we believe the dispute is correct).

 

Please let us know if you have any other questions or need any other clarification.

 

Thanks.

 

-Dan, for the Team

 

From: <[hidden email]> on behalf of jericho <[hidden email]>
Date: Monday, May 8, 2017 at 19:50
To: "Evans, Jonathan L." <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: RE: Current standards/criteria for 'Undefined Behavior'

 

Jonathan,

 

On Wed, 3 May 2017, Evans, Jonathan L. wrote:

 

: We are not entirely sure what you mean by "undefined behavior."  Are you

: talking about the CVE entries that specifically say "undefined

: behavior," such as CVE-2017-8326 and CVE-2017-7961?

 

Yes. A short list:

 

<a href="https://www.google.com/search?q=site%3Acve.mitre.org&#43;%22undefined&#43;behavior%22&amp;oq=site%3Acve.mitre.org&#43;%22undefined&#43;behavior%22&amp;aqs=chrome..69i57j69i58.3867j0j7&amp;sourceid=chrome&amp;ie=UTF-8" target="_blank">https://www.google.com/search?q=site%3Acve.mitre.org+%22undefined+behavior%22&oq=site%3Acve.mitre.org+%22undefined+behavior%22&aqs=chrome..69i57j69i58.3867j0j7&sourceid=chrome&ie=UTF-8

 

Thank you,

 

Brian

 

 

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Current standards/criteria for 'Undefined Behavior'

Landfield, Kent

That brings up a conflict for potentially a number of us.  The NIST Cybersecurity Framework Development Workshop is being held at NIST on the 16th and 17th.  I will not be available to participate in the CVE Board call since I will be attending and on a panel.  There may be others as well.  Any chance we can either push the call next week a day or two or push it a week to the following Wednesday? 

 

If we are going to be discussing the current thread, I definitely want to be there.

 

--

Kent Landfield

817-637-8026

[hidden email]

 

From: <[hidden email]> on behalf of "Levendis, Chris" <[hidden email]>
Date: Thursday, May 11, 2017 at 4:23 PM
To: "Millar, Thomas" <[hidden email]>, Carsten Eiram <[hidden email]>, "Adinolfi, Daniel R" <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: RE: Current standards/criteria for 'Undefined Behavior'

 

We're making it an agenda item for the next Board call.

 

Chris Levendis



Sent with BlackBerry Work
(www.blackberry.com)

From: Millar, Thomas <[hidden email]>

Date: Thursday, May 11, 2017, 4:45 PM

To: Carsten Eiram <[hidden email]>, Adinolfi, Daniel R <[hidden email]>

Cc: cve-editorial-board-list <[hidden email]>

Subject: RE: Current standards/criteria for 'Undefined Behavior'

 

I tend to agree with Carsten here but I assume a response further explaining the CVE Team’s reasoning is being drafted already.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Carsten Eiram
Sent: 11 May, 2017 07:19
To: Adinolfi, Daniel R <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: Re: Current standards/criteria for 'Undefined Behavior'

 

I hope the new MITRE CVE team realizes they are in a minority of people in this industry, who actually consider such issues as being CVE worthy by default or even security-relevant without some proof of there being a (realistic) security impact.

 

Referencing Hanno Böck and Agostino Sarubbo of all people does not lend a lot of "street cred" or underline the point. These guys are fairly inexperienced when it comes to vulnerability research and focus on fuzzing with AFL. One of them (Hanno) until recently didn't even really understand the output; the other still struggles.

 

We do not disagree that issues leading to undefined behaviour _theoretically_ have a security impact. Rarely is it ever proven, though. In fact, I don't think Agostino Sarubbo (or Hanno for that matter) has proven a single of the UBSan issues, which he has reported many of, actually did have a real-world impact.

 

Most experienced people in this industry have historically expected some reasonable indication of a security impact that was not too theoretical. There is a reason for us not seeing these types of reports from experienced researchers, who are also fuzzing using AFL. Even the OSS-Fuzz project flags UBSan issues as "Bug" and not "Bug-Security" by default. Sure, they should be fixed for good measure and are bad coding practice, but even developers generally don't treat the fixes as security-relevant. And rightly so.

 

MITRE is humoring beginners, who are fuzzing and just dumping their UBSan output without doing minimal due diligence; it would be better to educate them to higher standards i.e. the standards of most vulnerability researchers as well as the old MITRE CVE team. Otherwise, I hope the new MITRE CVE team is wearing their track pants and are on top of their cardio. You'll need some excellent stamina when taking on the CVE assignments for the metric ton of UBSan issues you'll be dealing with, if that is your default position.

 

/Carsten

 

 

On Wed, May 10, 2017 at 4:04 PM, Adinolfi, Daniel R <[hidden email]> wrote:

All,

 

The CVE Team largely agrees with community discussion of this topic at:

  http://openwall.com/lists/oss-security/2017/04/12/3

  http://openwall.com/lists/oss-security/2017/04/12/2

In terms of the CNA Rules, we feel that these undefined-behavior reports satisfy the definition of a vulnerability (Appendix A) because it is unreasonable for computational logic to depend on the implementation details of a compiler or the selected compiler options.

All, or nearly all, of the recent "undefined behavior" CVE IDs are the result of a discloser relying on the UndefinedBehaviorSanitizer tool that has design goals expressed in the http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html post. Also, these CVE IDs are for software products that are libraries, usable in building arbitrary programs including long-running daemons.

Referring to the C standard at http://www.open-std.org/jtc1/sc22/wg14/www/standards.html or similar online versions such as the http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf document, one can see statements such as "Possible undefined behavior ranges from ignoring the situation completely with unpredictable results ... to terminating a translation or execution." Terminating an execution is probably the most relevant for these CVE IDs. Also, the document

specifically mentions types of undefined behavior for which CVE IDs have been assigned, e.g., "If the value being converted is outside the range of values that can be represented, the behavior is undefined" and "The result of E1 << E2 is E1 left-shifted E2 bit positions ... otherwise, the behavior is undefined."

In other words, a program termination (aka a crash) is an outcome called out in the C standard. A crash within library code has a loss of availability, and thus matches the definition of a vulnerability.

 

One scenario of interest is a user who explicitly chooses compiler options that terminate the program upon encountering undefined behavior, e.g., "clang -fsanitize=undefined -fno-sanitize-recover" with LLVM. This means that an availability impact does occur. Admittedly, the availability impact may be rare in the real world, because people don't often use those clang flags for production code.

Another scenario of interest is compiler optimizations that happen to result in generation of vulnerable code for a program that relies on undefined behavior, such as CVE-2017-7892 which is about undefined behavior in "pointer arithmetic calculations that overflow." Here, the impact might not be limited to availability.

Based on the above, the MITRE CNA currently uses this guideline: a report of undefined behavior in a library is considered equivalent to a report of a crash in a library, in determining whether a CVE ID may exist.

However, Appendix E of the CNA Rules is also followed. For example, in the CVE-2017-7961 case, we received a dispute stating that there is no impact. We recorded the dispute as required (this does not necessarily mean that we believe the dispute is correct).

 

Please let us know if you have any other questions or need any other clarification.

 

Thanks.

 

-Dan, for the Team

 

From: <[hidden email]> on behalf of jericho <[hidden email]>
Date: Monday, May 8, 2017 at 19:50
To: "Evans, Jonathan L." <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: RE: Current standards/criteria for 'Undefined Behavior'

 

Jonathan,

 

On Wed, 3 May 2017, Evans, Jonathan L. wrote:

 

: We are not entirely sure what you mean by "undefined behavior."  Are you

: talking about the CVE entries that specifically say "undefined

: behavior," such as CVE-2017-8326 and CVE-2017-7961?

 

Yes. A short list:

 

<a href="https://www.google.com/search?q=site%3Acve.mitre.org&#43;%22undefined&#43;behavior%22&amp;oq=site%3Acve.mitre.org&#43;%22undefined&#43;behavior%22&amp;aqs=chrome..69i57j69i58.3867j0j7&amp;sourceid=chrome&amp;ie=UTF-8" target="_blank">https://www.google.com/search?q=site%3Acve.mitre.org+%22undefined+behavior%22&oq=site%3Acve.mitre.org+%22undefined+behavior%22&aqs=chrome..69i57j69i58.3867j0j7&sourceid=chrome&ie=UTF-8

 

Thank you,

 

Brian

 

 

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Current standards/criteria for 'Undefined Behavior'

Coffin, Chris

Kent,

 

Sorry… I don’t think we were aware of the conflict. My thinking is that we push to the following Wednesday as I would imagine that this already fits most folks schedules. Unless there are any objections, we’ll move the meeting early next week.

 

Chris

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Landfield, Kent
Sent: Thursday, May 11, 2017 4:42 PM
To: cve-editorial-board-list <[hidden email]>
Subject: Re: Current standards/criteria for 'Undefined Behavior'

 

That brings up a conflict for potentially a number of us.  The NIST Cybersecurity Framework Development Workshop is being held at NIST on the 16th and 17th.  I will not be available to participate in the CVE Board call since I will be attending and on a panel.  There may be others as well.  Any chance we can either push the call next week a day or two or push it a week to the following Wednesday? 

 

If we are going to be discussing the current thread, I definitely want to be there.

 

--

Kent Landfield

817-637-8026

[hidden email]

 

From: <[hidden email]> on behalf of "Levendis, Chris" <[hidden email]>
Date: Thursday, May 11, 2017 at 4:23 PM
To: "Millar, Thomas" <[hidden email]>, Carsten Eiram <[hidden email]>, "Adinolfi, Daniel R" <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: RE: Current standards/criteria for 'Undefined Behavior'

 

We're making it an agenda item for the next Board call.

 

Chris Levendis



Sent with BlackBerry Work
(www.blackberry.com)

From: Millar, Thomas <[hidden email]>

Date: Thursday, May 11, 2017, 4:45 PM

To: Carsten Eiram <[hidden email]>, Adinolfi, Daniel R <[hidden email]>

Cc: cve-editorial-board-list <[hidden email]>

Subject: RE: Current standards/criteria for 'Undefined Behavior'

 

I tend to agree with Carsten here but I assume a response further explaining the CVE Team’s reasoning is being drafted already.

 

From: [hidden email] [[hidden email]] On Behalf Of Carsten Eiram
Sent: 11 May, 2017 07:19
To: Adinolfi, Daniel R <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: Re: Current standards/criteria for 'Undefined Behavior'

 

I hope the new MITRE CVE team realizes they are in a minority of people in this industry, who actually consider such issues as being CVE worthy by default or even security-relevant without some proof of there being a (realistic) security impact.

 

Referencing Hanno Böck and Agostino Sarubbo of all people does not lend a lot of "street cred" or underline the point. These guys are fairly inexperienced when it comes to vulnerability research and focus on fuzzing with AFL. One of them (Hanno) until recently didn't even really understand the output; the other still struggles.

 

We do not disagree that issues leading to undefined behaviour _theoretically_ have a security impact. Rarely is it ever proven, though. In fact, I don't think Agostino Sarubbo (or Hanno for that matter) has proven a single of the UBSan issues, which he has reported many of, actually did have a real-world impact.

 

Most experienced people in this industry have historically expected some reasonable indication of a security impact that was not too theoretical. There is a reason for us not seeing these types of reports from experienced researchers, who are also fuzzing using AFL. Even the OSS-Fuzz project flags UBSan issues as "Bug" and not "Bug-Security" by default. Sure, they should be fixed for good measure and are bad coding practice, but even developers generally don't treat the fixes as security-relevant. And rightly so.

 

MITRE is humoring beginners, who are fuzzing and just dumping their UBSan output without doing minimal due diligence; it would be better to educate them to higher standards i.e. the standards of most vulnerability researchers as well as the old MITRE CVE team. Otherwise, I hope the new MITRE CVE team is wearing their track pants and are on top of their cardio. You'll need some excellent stamina when taking on the CVE assignments for the metric ton of UBSan issues you'll be dealing with, if that is your default position.

 

/Carsten

 

 

On Wed, May 10, 2017 at 4:04 PM, Adinolfi, Daniel R <[hidden email]> wrote:

All,

 

The CVE Team largely agrees with community discussion of this topic at:

  http://openwall.com/lists/oss-security/2017/04/12/3

  http://openwall.com/lists/oss-security/2017/04/12/2

In terms of the CNA Rules, we feel that these undefined-behavior reports satisfy the definition of a vulnerability (Appendix A) because it is unreasonable for computational logic to depend on the implementation details of a compiler or the selected compiler options.

All, or nearly all, of the recent "undefined behavior" CVE IDs are the result of a discloser relying on the UndefinedBehaviorSanitizer tool that has design goals expressed in the http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html post. Also, these CVE IDs are for software products that are libraries, usable in building arbitrary programs including long-running daemons.

Referring to the C standard at http://www.open-std.org/jtc1/sc22/wg14/www/standards.html or similar online versions such as the http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf document, one can see statements such as "Possible undefined behavior ranges from ignoring the situation completely with unpredictable results ... to terminating a translation or execution." Terminating an execution is probably the most relevant for these CVE IDs. Also, the document

specifically mentions types of undefined behavior for which CVE IDs have been assigned, e.g., "If the value being converted is outside the range of values that can be represented, the behavior is undefined" and "The result of E1 << E2 is E1 left-shifted E2 bit positions ... otherwise, the behavior is undefined."

In other words, a program termination (aka a crash) is an outcome called out in the C standard. A crash within library code has a loss of availability, and thus matches the definition of a vulnerability.

 

One scenario of interest is a user who explicitly chooses compiler options that terminate the program upon encountering undefined behavior, e.g., "clang -fsanitize=undefined -fno-sanitize-recover" with LLVM. This means that an availability impact does occur. Admittedly, the availability impact may be rare in the real world, because people don't often use those clang flags for production code.

Another scenario of interest is compiler optimizations that happen to result in generation of vulnerable code for a program that relies on undefined behavior, such as CVE-2017-7892 which is about undefined behavior in "pointer arithmetic calculations that overflow." Here, the impact might not be limited to availability.

Based on the above, the MITRE CNA currently uses this guideline: a report of undefined behavior in a library is considered equivalent to a report of a crash in a library, in determining whether a CVE ID may exist.

However, Appendix E of the CNA Rules is also followed. For example, in the CVE-2017-7961 case, we received a dispute stating that there is no impact. We recorded the dispute as required (this does not necessarily mean that we believe the dispute is correct).

 

Please let us know if you have any other questions or need any other clarification.

 

Thanks.

 

-Dan, for the Team

 

From: <[hidden email]> on behalf of jericho <[hidden email]>
Date: Monday, May 8, 2017 at 19:50
To: "Evans, Jonathan L." <[hidden email]>
Cc: cve-editorial-board-list <[hidden email]>
Subject: RE: Current standards/criteria for 'Undefined Behavior'

 

Jonathan,

 

On Wed, 3 May 2017, Evans, Jonathan L. wrote:

 

: We are not entirely sure what you mean by "undefined behavior."  Are you

: talking about the CVE entries that specifically say "undefined

: behavior," such as CVE-2017-8326 and CVE-2017-7961?

 

Yes. A short list:

 

<a href="https://www.google.com/search?q=site%3Acve.mitre.org&#43;%22undefined&#43;behavior%22&amp;oq=site%3Acve.mitre.org&#43;%22undefined&#43;behavior%22&amp;aqs=chrome..69i57j69i58.3867j0j7&amp;sourceid=chrome&amp;ie=UTF-8" target="_blank">https://www.google.com/search?q=site%3Acve.mitre.org+%22undefined+behavior%22&oq=site%3Acve.mitre.org+%22undefined+behavior%22&aqs=chrome..69i57j69i58.3867j0j7&sourceid=chrome&ie=UTF-8

 

Thank you,

 

Brian

 

 

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Current standards/criteria for 'Undefined Behavior'

jericho
On Thu, 11 May 2017, Coffin, Chris wrote:

: Sorry? I don?t think we were aware of the conflict. My thinking is that
: we push to the following Wednesday as I would imagine that this already
: fits most folks schedules. Unless there are any objections, we?ll move
: the meeting early next week.

I'd also like to remind MITRE that the calls do not include all of the
board for a variety of reasons. I haven't looked at the notes lately to
see if there is even a majority.

But two of the people who brought this up / showed concern and gave reason
for MITRE to reconsider, are typically not on those calls.

As such, I think this is a great time to have a discussion on the list
where we can all participate and have it documented fully.

Brian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Current standards/criteria for 'Undefined Behavior'

Landfield, Kent
Can you let us know why you don’t participate in the calls? Is it a time issue? If so we can work to try to find a better time that accommodates more Board members.  I agree and have stated in the past that real decisions need to be made on the Board list(s).  The Board calls however, do give us a higher bandwidth opportunity to go more in-depth on specific issues.  We need all to be there if possible and have had reasonable attendance lately.  

Can you enlighten us as to why you do not attend?

--
Kent Landfield
817-637-8026
[hidden email]

On 5/11/17, 11:52 PM, "[hidden email] on behalf of jericho" <[hidden email] on behalf of [hidden email]> wrote:

    On Thu, 11 May 2017, Coffin, Chris wrote:
   
    : Sorry? I don?t think we were aware of the conflict. My thinking is that
    : we push to the following Wednesday as I would imagine that this already
    : fits most folks schedules. Unless there are any objections, we?ll move
    : the meeting early next week.
   
    I'd also like to remind MITRE that the calls do not include all of the
    board for a variety of reasons. I haven't looked at the notes lately to
    see if there is even a majority.
   
    But two of the people who brought this up / showed concern and gave reason
    for MITRE to reconsider, are typically not on those calls.
   
    As such, I think this is a great time to have a discussion on the list
    where we can all participate and have it documented fully.
   
    Brian
   

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Current standards/criteria for 'Undefined Behavior'

jericho
On Fri, 12 May 2017, Landfield, Kent wrote:

: Can you let us know why you don?t participate in the calls? Is it a time

Before I do that, can you give me a summary of the calls since they
started, as far as the % of Board members that attend? What is the average
or medium for attendance? If you can't, you should probably stop to
consider if these calls were a vehicle for MITRE to usurp control in some
fashion. After all, they have randomly usurped control on so many other
critical / industry-shocking changes, without our review. Remind me why we
trust MITRE at this point? Stop considering them "your fellow admin", and
start considering them as "APTderp". I think that might be a better
analogy and more prudent.

: issue? If so we can work to try to find a better time that accommodates
: more Board members.  I agree and have stated in the past that real

Given the current Board, and I am fairly sure we went through this for
weeks... trying to find a time that works for EVERYONE is a lost cause.
The current time was selected based on the "best we could do", no? I think
we have some mails archived on this.

: decisions need to be made on the Board list(s).  The Board calls
: however, do give us a higher bandwidth opportunity to go more in-depth
: on specific issues.  We need all to be there if possible and have had

They do. But until we have a true transcript of those calls, and the calls
are treated as a "single email" in the context of the Board, it simply
isn't fair. Decisions are effectively made on these calls without the
consent of the board.

: Can you enlighten us as to why you do not attend?

Sure! You can guess which is more important to me:

1. I am typically not available Thursday at ~ 1PM or whenver they were. I
deleted my Calender event because I was basically never available (best
case, I was driving up I-70 through dead zones and the tunnels, which i
spent a year working with a local T-Mobile managing engineer to resolve).
I can also guarantee you, that the Europeans will never make that time
unless they stay up VERY late, after a 14 hour day working, often fighting
to understand horrible CVE assignments.

2. We get a rough summary of the call, but not real detail. We get
"minutes", great. That doesn't tell me "Kent was really worked up, and
thought that $newidea was complete crap". It doesn't tell me that
"$whoever objected quite a bit", or what was said to resolve it and
ultimately make some "informed" decision.

3. I have long had a serious disdain for InfoSec people who insisted on
phone calls, after a few emails. In my personal experience, after too many
years, they did it because they specifically did NOT want a record.
Usually because they were trying to explain why they weren't a charlatan /
fraud, and why you could clearly trust them as a human. [Disclaimer:
remember, I was the primary person behind Attrition Errata.]

4. Based on the above, security is about integrity. We're auditors. We
like logs... records... a transcript of what transpired. Until I have
that, and understand where a conclusion came from? I don't consider myself
informed. Don't in turn expect me to make an informed vote on anything.

.b
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Current standards/criteria for 'Undefined Behavior'

kseifried@redhat.com
On 13/05/17 12:28 AM, jericho wrote:

> On Fri, 12 May 2017, Landfield, Kent wrote:
>
> : Can you let us know why you don?t participate in the calls? Is it a time
>
> Before I do that, can you give me a summary of the calls since they
> started, as far as the % of Board members that attend? What is the average
> or medium for attendance? If you can't, you should probably stop to
> consider if these calls were a vehicle for MITRE to usurp control in some
> fashion. After all, they have randomly usurped control on so many other
> critical / industry-shocking changes, without our review. Remind me why we
> trust MITRE at this point? Stop considering them "your fellow admin", and
> start considering them as "APTderp". I think that might be a better
> analogy and more prudent.

Sorry I'm having a hard time understanding how board calls, that the
entire board is invited to, and are held at a reasonable
time/availability and involve quite a lot of board members (e.g. myself
and Kent =) holding MITRE's feet to the fire as it were as an attempt by
MITRE to usurp control (for uh.. something that they already control...
at least as I understand the DHS funding/MITRE/etc stuff).

>
> : issue? If so we can work to try to find a better time that accommodates
> : more Board members.  I agree and have stated in the past that real
>
> Given the current Board, and I am fairly sure we went through this for
> weeks... trying to find a time that works for EVERYONE is a lost cause.
> The current time was selected based on the "best we could do", no? I think
> we have some mails archived on this.

Yup, and to that end we have the board mailing list, minutes of the
calls, etc. It's like embargo release times, it's mostly a case of "what
is the least worst time".

>
> : decisions need to be made on the Board list(s).  The Board calls
> : however, do give us a higher bandwidth opportunity to go more in-depth
> : on specific issues.  We need all to be there if possible and have had
>
> They do. But until we have a true transcript of those calls, and the calls
> are treated as a "single email" in the context of the Board, it simply
> isn't fair. Decisions are effectively made on these calls without the
> consent of the board.

I personally wouldn't be comfortable with a word for word transcript of
the call as myself and quite a few others often speak off the cuff (and
say things in a way that might not be politic, or intelligent sounding,
but in summary seem rational).

>
> : Can you enlighten us as to why you do not attend?
>
> Sure! You can guess which is more important to me:

Well there you go, CVE is important enough to you for emails, but not
board calls. I and I bet everyone on the board is also busy. But we make
time for this stuff (hint: the board calls actually cross over into time
with my autistic son, but I make it work because my personal life and my
professional life are both important).


> 1. I am typically not available Thursday at ~ 1PM or whenver they were. I
> deleted my Calender event because I was basically never available (best
> case, I was driving up I-70 through dead zones and the tunnels, which i
> spent a year working with a local T-Mobile managing engineer to resolve).
> I can also guarantee you, that the Europeans will never make that time
> unless they stay up VERY late, after a 14 hour day working, often fighting
> to understand horrible CVE assignments.
>
> 2. We get a rough summary of the call, but not real detail. We get
> "minutes", great. That doesn't tell me "Kent was really worked up, and
> thought that $newidea was complete crap". It doesn't tell me that
> "$whoever objected quite a bit", or what was said to resolve it and
> ultimately make some "informed" decision.
>
> 3. I have long had a serious disdain for InfoSec people who insisted on
> phone calls, after a few emails. In my personal experience, after too many
> years, they did it because they specifically did NOT want a record.
> Usually because they were trying to explain why they weren't a charlatan /
> fraud, and why you could clearly trust them as a human. [Disclaimer:
> remember, I was the primary person behind Attrition Errata.]
>
> 4. Based on the above, security is about integrity. We're auditors. We
> like logs... records... a transcript of what transpired. Until I have
> that, and understand where a conclusion came from? I don't consider myself
> informed. Don't in turn expect me to make an informed vote on anything.

Or you could just hop on the call.

>
> .b
>

--

Kurt Seifried -- Red Hat -- Product Security -- Cloud
PGP A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
Red Hat Product Security contact: [hidden email]
Loading...