Protocol Buffers and forward compatibility with enums broken?

Question

I was looking into protobuf and enum and trying to understand forward compatibility. So I tested the following:

I have this proto file:

syntax = "proto3";
package com.transport;

message CustomerOptions {
  int32 id = 1;
  TransportOptions transport_option = 2;
}

enum TransportOptions {
  UNKNOWN = 0;
  CAR = 1;
  BUS = 2;
  TRAIN = 3;
  AIRPLANE = 4;
}

What I did was save that to a binary file as follows:

CustomerOptions.Builder builder1 = CustomerOptions.newBuilder();
builder1.setId(1);
builder1.setTransportOption(Options.TransportOptions.AIRPLANE);
CustomerOptions options = builder1.build();

FileOutputStream fileOutputStream = new FileOutputStream("customer_options_message.bin");
options.writeTo(fileOutputStream);

Now what I did was remove a field from the TransportOptions in the proto file which now became:

syntax = "proto3";
package com.transport;

message CustomerOptions {
  int32 id = 1;
  TransportOptions transport_option = 2;
}

enum TransportOptions {
  UNKNOWN = 0;
  CAR = 1;
  BUS = 2;
  TRAIN = 3;
}

So now I removed AIRPLANE.

Now with this new schema I read the binary file that was written before that had AIRPLANE as follows:

FileInputStream fileInputStream = new FileInputStream("customer_options_message.bin");
CustomerOptions customerOptions = CustomerOptions.parseFrom(fileInputStream);  

Now what happens is the following:

If I do:

System.out.println(customerOptions.getTransportOption());  

this prints:

UNRECOGNIZED

I was expecting that it would print UKNOWN i.e that since the code reads an unknown field (AIRPLANE) it would use the default value which should be the first one i.e. UNKNOWN = 0.

Why is this not the case.
The weird is that if I do the following:

System.out.println(customerOptions);  

I get:

transport_option: UNKNOWN_ENUM_VALUE_TransportOptions_4

So again I see UNKNOWN but it seems that the value is -1 and not 0.

What exactly is going on here? How does compatibility work with enums?

Answer 1

The Updating a Message Type section in proto3 language guide says:

unrecognized proto3 enum types will be preserved in the message, but how this is represented when the message is deserialized is language-dependent.

The Java Generated Code Reference further says:

If you are using proto3, it also adds the special value UNRECOGNIZED to the enum type.

So it appears this is the way the Java protobuf library is designed to represent unknown enum values.