Skip to main content
 首页 » 编程设计

python之unpack_from 需要至少 4 个字节的缓冲区

2025年05月04日10telwanggs

我正在从客户端接收到一个数据包,其中包含许多字段。我成功读取了所有字段,但是当涉及到最后一个字段 tag_end 时,python 给我一个错误:

unpack_from requires a buffer of at least 4 bytes not found.

这是代码:

def set_bin(self, buf): 
    """Reads a vector of bytes (probably received from network or 
    read from file) and tries to construct the packet structure 
    from it, by reading each packet member from the buffer.  This 
    is somehow like deserializing the packet. 
    """ 
    assert isinstance(buf, bytearray), 'buffer type is not valid' 
    offset = 0 
 
    print("$$$$$$$$$$$$$$$$ set bin $$$$$$$$$$$$$$$$$") 
 
    try: 
        (self._tag_start, self._version, self._checksum, self._connection_id, 
         self._packet_seq) = Packet.PACKER_1.unpack_from(str(buf), offset) 
    except struct.error as e: 
        print(e) 
        raise DeserializeError(e) 
    except ValueError as e: 
        print(e) 
        raise DeserializeError(e) 
          #I=4 H=2 B=1 
    offset = Packet.OFFSET_GUID     #14 correct 
    self._guid = buf[offset:offset+Packet.UUID_SIZE] #14-16 correct 
 
    offset = Packet.OFFSET_GUID + Packet.UUID_SIZE 
 
    print("$$$$$$$$$$$$$$$$ GUID read successfully  $$$$$$$$$$$$$$$$$") 
 
    try: 
       (self._timestamp_sec, self._timestamp_microsec, self._command, 
        self._command_seq, self._subcommand, self._data_seq, 
        self._data_length) = Packet.PACKER_3.unpack_from(str(buf), offset) 
    except struct.error as e: 
        print(e) 
        raise DeserializeError(e) 
    except ValueError as e: 
        print(e) 
        raise DeserializeError(e) 
 
    print("$$$$$$$$$$$$$$$$ timestamps read successfully $$$$$$$$$$$$$$$$$") 
   offset = Packet.OFFSET_AUTHENTICATE 
    self._username = buf[offset:offset + self.USERNAME_SIZE]        #Saman 
    offset += self.USERNAME_SIZE 
 
    print("$$$$$$$$$$$$$$$$ username read successfully  $$$$$$$$$$$$$$$$$") 
 
    self._password = buf[offset:offset+self.USERNAME_SIZE] 
    offset += self.PASSWORD_SIZE 
 
    print("$$$$$$$$$$$$$$$$ password read successfully $$$$$$$$$$$$$$$$$") 
 
    self._data = buf[offset:offset+self._data_length] 
    offset = offset + self._data_length 
 
    print("$$$$$$$$$$$$$$$$ data read successfully  $$$$$$$$$$$$$$$$$") 
 
    try: 
        (self._tag_end,) = Packet.PACKER_4.unpack_from(str(buf), offset) 
    except struct.error as e: 
        print(e) 
        raise DeserializeError(e) 
    except ValueError as e: 
        print(e) 
        raise DeserializeError(e) 
 
    print("$$$$$$$$$$$$$$$$ tag end read successfully $$$$$$$$$$$$$$$$$") 
 
 
    if len(buf) != Packet.PACKER.size + self._data_length: 
        print('failed to deserialize binary data correctly and construct the packet due to extra data') 
    else: 
        print('@@@@@@@@@@@@@@@ Deserialized Successfully') 

这是代码中使用的一些常量:

STRUCT_FORMAT_STR   = r'=IHIHH 16B IIHHHHH I 6c 9c' #Saman 
STRUCT_FORMAT_STR_1 = r'=IHIHH' 
STRUCT_FORMAT_STR_2 = r'=16B' 
STRUCT_FORMAT_STR_3 = r'=IIHHHHH' 
STRUCT_FORMAT_STR_4 = r'=I' 
STRUCT_FORMAT_STR_5 = r'=6c' 
STRUCT_FORMAT_STR_6 = r'=9c' 
UUID_SIZE   = 16 
OFFSET_GUID = 14 
#OFFSET_DATA = 48     #shifting offset data by 15 char       
OFFSET_AUTHENTICATE = 48 
PACKER   = struct.Struct(str(STRUCT_FORMAT_STR))     #Saman 
PACKER_1 = struct.Struct(str(STRUCT_FORMAT_STR_1)) 
PACKER_2 = struct.Struct(str(STRUCT_FORMAT_STR_2)) 
PACKER_3 = struct.Struct(str(STRUCT_FORMAT_STR_3)) 
PACKER_4 = struct.Struct(str(STRUCT_FORMAT_STR_4)) 
PACKER_5 = struct.Struct(str(STRUCT_FORMAT_STR_5)) 
PACKER_6 = struct.Struct(str(STRUCT_FORMAT_STR_6)) 
BYTES_TAG_START = PACKER_4.pack(TAG_START) 
BYTES_TAG_END   = PACKER_4.pack(TAG_END) 

和数据包对象的初始化,它在其中初始化字段:

def init(self, **kwargs): 如果'buf'在kwargs中: self.set_bin(kwargs['buf']) 别的: 断言 Packet.RTCINET_COMMANDS.values() 中的 kwargs['command'] 和 Packet.RTCINET_COMMANDS.values() 中的 kwargs['subcommand'], 'Undefined protocol command' assert isinstance(kwargs['data'], bytearray), '数据字段类型无效' 对于('命令','子命令','数据')中的字段: setattr(self, '_' + field, kwargs[field])

    self._tag_start = Packet.TAG_START 
    self._version = Packet.VERSION_CURRENT % (Packet.USHRT_MAX + 1) 
    self._checksum = Packet.CRC_INIT 
    self._connection_id = kwargs.get('connection_id', 0) % (Packet.USHRT_MAX + 1) 
    self._packet_seq = Packet.PACKET_SEQ 
    Packet.PACKET_SEQ = (Packet.PACKET_SEQ + 1) % (Packet.USHRT_MAX + 1) 
    self._guid = uuid.uuid4().bytes 
    dt = datetime.datetime.now() 
    self._timestamp_sec = int(time.mktime(dt.timetuple())) 
    self._timestamp_microsec = dt.microsecond 
    # self._command = kwargs['command'] 
    self._command_seq = kwargs.get('command_seq', 0) 
    # self._subcommand = kwargs['subcommand'] 
    self._data_seq = kwargs.get('data_seq', 0) 
    self._data_length = len(kwargs['data']) 
 
    self._username = Packet.USERNAME            #Saman 
    self._password = Packet.PASSWORD 

我已确保以正确的顺序读取所有字段,因为它是由客户端程序写入数据包中的。但我仍然无法解决这个问题。

你知道如何解决这个问题吗?

请您参考如下方法:

问题似乎是您无缘无故地将所有内容转换为 str。

在某些地方,例如 PACKER_1 = struct.Struct(str(STRUCT_FORMAT_STR_1)),它会降低您的代码的可读性和可理解性,但不会影响实际输出。例如,STRUCT_FORMAT_STR_1 已经是一个 str,所以 str(STRUCT_FORMAT_STR_1) 是相同的 str。

但在其他地方,情况远比这糟糕。特别是,查看所有类似 Packet.PACKER_1.unpack_from(str(buf), offset) 的行。其中,buf 是一个bytearray。 (它必须是,因为你 assert 它。)在 bytearray 上调用 str 给你那个 bytearray< 的字符串表示。例如:

>>> b = bytearray(b'abc') 
>>> len(b) 
3 
>>> s = str(b) 
>>> s 
"bytearray(b'abc')" 
>>> len(s) 
17 

该字符串表示显然通常不会与您所表示的实际缓冲区具有相同的长度。所以难怪你会得到关于长度错误的错误。 (如果你真的不走运并且没有任何这样的错误,你就会读取垃圾值。)

那么,您应该如何将 bytearray 转换为 struct 模块可以处理的内容?没有什么!作为the docs说:

Several struct functions (and methods of Struct) take a buffer argument. This refers to objects that implement the Buffer Protocol and provide either a readable or read-writable buffer. The most common types used for that purpose are bytes and bytearray…