Is it possible to receive messages on a topic which contain out-of-bound values in message fields?

Question

As far as I know it is not possible to publish out-of-bound values for message fields with ROS nodes/nodelets. E.g. the only possible publish-able range for uint8 message field values is in the range between 0 and 2^8-1. But is it possible to inject out-of-bound values in the full data range of the underlying data type of the ROS client implementation into a message published on a topic? That a subscriber (ROS Python client) to a topic would receive messages with e.g. a message field type of uint8 in the full range of integer data type. Or do clients silently validate and drop such messages if received?

---

Originally posted by thinwybk on ROS Answers with karma: 468 on 2018-06-01

Post score: 0

---

Original comments

Comment by jayess on 2018-06-01:
Just a pedantic nitpick, but the range for uint8 is actually 2^8 - 1 = 255, while 2*8 - 1 = 31. So, either this is a typo or if you're publishing values in the range [32, 255] you're actually still in range for uint8.

Comment by thinwybk on 2018-06-01:
Right. Python 2**8-1. fixed

Comment by gvdhoorn on 2018-06-02:
Another nitpick: are we mixing up the concepts "out of band messaging" and an "out of bounds" problem when representing a value? The former may be possible, depending on how strict you define it and map that to ROS infrastructure. The latter makes a little less sense to me in this context.

Comment by thinwybk on 2018-06-02:
I am not sure if I understand "out of band messaging" in the ROS context. It is about message field values which are inside the range of the underlying Python data type (e.g. Python int) but outside the corresponding ROS data type (e.g. ROS uint8). (E.g. here int with wider value range as uint8).

Comment by gvdhoorn on 2018-06-02:
Your question title says "receive messages [..] which are out-of-bound". That doesn't make sense to me. Fields could potentially have values in the serialised bytestream that are out-of-bounds for their specced types, but this statement cannot apply to complete messages ..

Comment by thinwybk on 2018-06-02:
You are right. I adjusted the question accordingly. :) Sry for the confusion.

Answer 1

(this is more of an addition to @ahendrix's answer (but I didn't want to edit his))

See the (de)serialisation code for UInt8.msg for how this works in Python fi (taken from /opt/ros/kinetic/lib/python2.7/dist-packages/std_msgs/msg/_UInt8.py):

  def serialize(self, buff):
    """
    serialize message into buffer
    :param buff: buffer, ``StringIO``
    """
    try:
      buff.write(_get_struct_B().pack(self.data))
    except struct.error as se: self._check_types(struct.error("%s: '%s' when writing '%s'" % (type(se), str(se), str(locals().get('_x', self)))))
    except TypeError as te: self._check_types(ValueError("%s: '%s' when writing '%s'" % (type(te), str(te), str(locals().get('_x', self)))))

  def deserialize(self, str):
    """
    unpack serialized message in str into this message instance
    :param str: byte array of serialized message, ``str``
    """
    try:
      end = 0
      start = end
      end += 1
      (self.data,) = _get_struct_B().unpack(str[start:end])
      return self
    except struct.error as e:
      raise genpy.DeserializationError(e) #most likely buffer underfill

At least at this level (and this is the ony place message content is actually (de)serialised, called from here and here), values that are unrepresentable by the type that was specced for a field in a message will result in exceptions being thrown.

@ahendrix wrote:

I believe Python is using the struct library to pack objects into the serialization format, so you will get an exception if the value in that message field is outside of the allowed bounds.

Exactly: _get_struct_B() essentially just returns an instance of struct.Struct("B"):

def _get_struct_B():
    global _struct_B
    if _struct_B is None:
        _struct_B = struct.Struct("<B")
    return _struct_B

Note also the endianness specification there.

---

Originally posted by gvdhoorn with karma: 86574 on 2018-06-02

This answer was ACCEPTED on the original site

Post score: 2

---

Original comments

Comment by thinwybk on 2018-06-02:
Thx. My question is about the deserialization part. Sry. I modified the question accordingly. About the background of the question: hypothesis-ros review issue 48

Comment by gvdhoorn on 2018-06-02:
I'm confused: the code snippets I copied show both serialisation as well as deserialisation. The same comments apply to both of them.

Comment by thinwybk on 2018-06-02:
Right now it is not clear to me if a topic subscriber implemented in Python could deserialize a message with a "hacked" message field of ROS uint8 which has a value in it which is > 2^8-1 (within the Python int value range). I could not find where the deserialize function validates this...

Comment by gvdhoorn on 2018-06-02:\

I could not find where the deserialize function validates this...

It doesn't. See the lines in deserialize(..). The only error that can occur is if there are not enough bytes for struct to unpack.

This is all the code there is (at least for Python).

Comment by thinwybk on 2018-06-02:
@gvdhoorn Thx a lot. And sorry for the confusion until here :)

Answer 2

The ROS serialization format only allocates enough bits for the advertised data size, so there cannot be out-of-range values in a serialized message.

I believe C++ will either give you a compiler warning or silently truncate your integers if you assign from a larger variable into a message field with less precision (this depends on your compiler and which warnings you have enabled).

I believe Python is using the struct library to pack objects into the serialization format, so you will get an exception if the value in that message field is outside of the allowed bounds.

---

Originally posted by ahendrix with karma: 47576 on 2018-06-02

This answer was NOT ACCEPTED on the original site

Post score: 3