前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >聊聊SpinalTap的Transaction

聊聊SpinalTap的Transaction

原创
作者头像
code4it
修改2020-06-02 10:29:30
5090
修改2020-06-02 10:29:30
举报
文章被收录于专栏:码匠的流水账码匠的流水账

本文主要研究一下SpinalTap的Transaction

Transaction

SpinalTap/spinaltap-model/src/main/java/com/airbnb/spinaltap/mysql/Transaction.java

代码语言:javascript
复制
@Value
@RequiredArgsConstructor
public class Transaction {
  private final long timestamp;
  private final long offset;
  private final BinlogFilePos position;
  private final String gtid;
?
  public Transaction(long timestamp, long offset, BinlogFilePos position) {
    this.timestamp = timestamp;
    this.offset = offset;
    this.position = position;
    this.gtid = null;
  }
}
  • Transaction定义了timestamp、offset、position、gtid属性

MysqlMutationMetadata

SpinalTap/spinaltap-model/src/main/java/com/airbnb/spinaltap/mysql/mutation/MysqlMutationMetadata.java

代码语言:javascript
复制
@Value
@ToString(callSuper = true)
@EqualsAndHashCode(callSuper = true)
public class MysqlMutationMetadata extends Mutation.Metadata {
  private final DataSource dataSource;
  private final BinlogFilePos filePos;
  private final Table table;
  private final long serverId;
  private final Transaction beginTransaction;
  private final Transaction lastTransaction;
?
  /** The leader epoch of the node resource processing the event. */
  private final long leaderEpoch;
?
  /** The mutation row position in the given binlog event. */
  private final int eventRowPosition;
?
  public MysqlMutationMetadata(
      DataSource dataSource,
      BinlogFilePos filePos,
      Table table,
      long serverId,
      long id,
      long timestamp,
      Transaction beginTransaction,
      Transaction lastTransaction,
      long leaderEpoch,
      int eventRowPosition) {
    super(id, timestamp);
?
    this.dataSource = dataSource;
    this.filePos = filePos;
    this.table = table;
    this.serverId = serverId;
    this.beginTransaction = beginTransaction;
    this.lastTransaction = lastTransaction;
    this.leaderEpoch = leaderEpoch;
    this.eventRowPosition = eventRowPosition;
  }
}
  • MysqlMutationMetadata定义了dataSource、filePos、table、serverId、beginTransaction、lastTransaction、leaderEpoch、eventRowPosition属性

MysqlMutationMapper

SpinalTap/spinaltap-mysql/src/main/java/com/airbnb/spinaltap/mysql/event/mapper/MysqlMutationMapper.java

代码语言:javascript
复制
@Slf4j
@RequiredArgsConstructor
public abstract class MysqlMutationMapper<R extends BinlogEvent, T extends MysqlMutation>
    implements Mapper<R, List<T>> {
  @NonNull private final DataSource dataSource;
  @NonNull private final TableCache tableCache;
  @NonNull private final AtomicReference<Transaction> beginTransaction;
  @NonNull private final AtomicReference<Transaction> lastTransaction;
  @NonNull private final AtomicLong leaderEpoch;
?
  public static Mapper<BinlogEvent, List<? extends Mutation<?>>> create(
      @NonNull final DataSource dataSource,
      @NonNull final TableCache tableCache,
      @NonNull final SchemaTracker schemaTracker,
      @NonNull final AtomicLong leaderEpoch,
      @NonNull final AtomicReference<Transaction> beginTransaction,
      @NonNull final AtomicReference<Transaction> lastTransaction,
      @NonNull final MysqlSourceMetrics metrics) {
    final AtomicReference<String> gtid = new AtomicReference<>();
    return ClassBasedMapper.<BinlogEvent, List<? extends Mutation<?>>>builder()
        .addMapper(TableMapEvent.class, new TableMapMapper(tableCache))
        .addMapper(GTIDEvent.class, new GTIDMapper(gtid))
        .addMapper(QueryEvent.class, new QueryMapper(beginTransaction, gtid, schemaTracker))
        .addMapper(XidEvent.class, new XidMapper(lastTransaction, gtid, metrics))
        .addMapper(StartEvent.class, new StartMapper(dataSource, tableCache, metrics))
        .addMapper(
            UpdateEvent.class,
            new UpdateMutationMapper(
                dataSource, tableCache, beginTransaction, lastTransaction, leaderEpoch))
        .addMapper(
            WriteEvent.class,
            new InsertMutationMapper(
                dataSource, tableCache, beginTransaction, lastTransaction, leaderEpoch))
        .addMapper(
            DeleteEvent.class,
            new DeleteMutationMapper(
                dataSource, tableCache, beginTransaction, lastTransaction, leaderEpoch))
        .build();
  }
?
  protected abstract List<T> mapEvent(@NonNull final Table table, @NonNull final R event);
?
  public List<T> map(@NonNull final R event) {
    Table table = tableCache.get(event.getTableId());
?
    return mapEvent(table, event);
  }
?
  MysqlMutationMetadata createMetadata(
      @NonNull final Table table, @NonNull final BinlogEvent event, final int eventPosition) {
    return new MysqlMutationMetadata(
        dataSource,
        event.getBinlogFilePos(),
        table,
        event.getServerId(),
        event.getOffset(),
        event.getTimestamp(),
        beginTransaction.get(),
        lastTransaction.get(),
        leaderEpoch.get(),
        eventPosition);
  }
?
  static ImmutableMap<String, Column> zip(
      @NonNull final Serializable[] row, @NonNull final Collection<ColumnMetadata> columns) {
    if (row.length != columns.size()) {
      log.error("Row length {} and column length {} don't match", row.length, columns.size());
    }
?
    final ImmutableMap.Builder<String, Column> builder = ImmutableMap.builder();
    final Iterator<ColumnMetadata> columnIterator = columns.iterator();
?
    for (int position = 0; position < row.length && columnIterator.hasNext(); position++) {
      final ColumnMetadata col = columnIterator.next();
      builder.put(col.getName(), new Column(col, row[position]));
    }
?
    return builder.build();
  }
}
  • MysqlMutationMapper定义了dataSource、tableCache、beginTransaction、lastTransaction、leaderEpoch属性;它提供了createMetadata方法,它接收table、event、eventPosition参数返回新建的MysqlMutationMetadata

InsertMutationMapper

SpinalTap/spinaltap-mysql/src/main/java/com/airbnb/spinaltap/mysql/event/mapper/InsertMutationMapper.java

代码语言:javascript
复制
class InsertMutationMapper extends MysqlMutationMapper<WriteEvent, MysqlInsertMutation> {
  InsertMutationMapper(
      @NonNull final DataSource dataSource,
      @NonNull final TableCache tableCache,
      @NonNull final AtomicReference<Transaction> beginTransaction,
      @NonNull final AtomicReference<Transaction> lastTransaction,
      @NonNull final AtomicLong leaderEpoch) {
    super(dataSource, tableCache, beginTransaction, lastTransaction, leaderEpoch);
  }
?
  @Override
  protected List<MysqlInsertMutation> mapEvent(
      @NonNull final Table table, @NonNull final WriteEvent event) {
    final List<Serializable[]> rows = event.getRows();
    final List<MysqlInsertMutation> mutations = new ArrayList<>();
    final Collection<ColumnMetadata> cols = table.getColumns().values();
?
    for (int position = 0; position < rows.size(); position++) {
      mutations.add(
          new MysqlInsertMutation(
              createMetadata(table, event, position),
              new Row(table, zip(rows.get(position), cols))));
    }
?
    return mutations;
  }
}
  • InsertMutationMapper继承了MysqlMutationMapper,其构造器要求输入dataSource、tableCache、beginTransaction、lastTransaction、leaderEpoch参数

UpdateMutationMapper

SpinalTap/spinaltap-mysql/src/main/java/com/airbnb/spinaltap/mysql/event/mapper/UpdateMutationMapper.java

代码语言:javascript
复制
final class UpdateMutationMapper extends MysqlMutationMapper<UpdateEvent, MysqlMutation> {
  UpdateMutationMapper(
      @NonNull final DataSource dataSource,
      @NonNull final TableCache tableCache,
      @NonNull final AtomicReference<Transaction> beginTransaction,
      @NonNull final AtomicReference<Transaction> lastTransaction,
      @NonNull final AtomicLong leaderEpoch) {
    super(dataSource, tableCache, beginTransaction, lastTransaction, leaderEpoch);
  }
?
  @Override
  protected List<MysqlMutation> mapEvent(
      @NonNull final Table table, @NonNull final UpdateEvent event) {
    final List<MysqlMutation> mutations = Lists.newArrayList();
    final Collection<ColumnMetadata> cols = table.getColumns().values();
    final List<Map.Entry<Serializable[], Serializable[]>> rows = event.getRows();
?
    for (int position = 0; position < rows.size(); position++) {
      MysqlMutationMetadata metadata = createMetadata(table, event, position);
?
      final Row previousRow = new Row(table, zip(rows.get(position).getKey(), cols));
      final Row newRow = new Row(table, zip(rows.get(position).getValue(), cols));
?
      // If PK value has changed, then delete before image and insert new image
      // to retain invariant that a mutation captures changes to a single PK
      if (table.getPrimaryKey().isPresent()
          && !previousRow.getPrimaryKeyValue().equals(newRow.getPrimaryKeyValue())) {
        mutations.add(new MysqlDeleteMutation(metadata, previousRow));
        mutations.add(new MysqlInsertMutation(metadata, newRow));
      } else {
        mutations.add(new MysqlUpdateMutation(metadata, previousRow, newRow));
      }
    }
?
    return mutations;
  }
}
  • UpdateMutationMapper继承了MysqlMutationMapper,其构造器要求输入dataSource、tableCache、beginTransaction、lastTransaction、leaderEpoch参数

DeleteMutationMapper

SpinalTap/spinaltap-mysql/src/main/java/com/airbnb/spinaltap/mysql/event/mapper/DeleteMutationMapper.java

代码语言:javascript
复制
final class DeleteMutationMapper extends MysqlMutationMapper<DeleteEvent, MysqlDeleteMutation> {
  DeleteMutationMapper(
      @NonNull final DataSource dataSource,
      @NonNull final TableCache tableCache,
      @NonNull final AtomicReference<Transaction> beginTransaction,
      @NonNull final AtomicReference<Transaction> lastTransaction,
      @NonNull final AtomicLong leaderEpoch) {
    super(dataSource, tableCache, beginTransaction, lastTransaction, leaderEpoch);
  }
?
  @Override
  protected List<MysqlDeleteMutation> mapEvent(
      @NonNull final Table table, @NonNull final DeleteEvent event) {
    final Collection<ColumnMetadata> cols = table.getColumns().values();
    final List<MysqlDeleteMutation> mutations = new ArrayList<>();
    final List<Serializable[]> rows = event.getRows();
?
    for (int position = 0; position < rows.size(); position++) {
      mutations.add(
          new MysqlDeleteMutation(
              createMetadata(table, event, position),
              new Row(table, zip(rows.get(position), cols))));
    }
?
    return mutations;
  }
}
  • DeleteMutationMapper继承了MysqlMutationMapper,其构造器要求输入dataSource、tableCache、beginTransaction、lastTransaction、leaderEpoch参数

小结

Transaction定义了timestamp、offset、position、gtid属性;MysqlMutationMetadata定义了dataSource、filePos、table、serverId、beginTransaction、lastTransaction、leaderEpoch、eventRowPosition属性

doc

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • Transaction
  • MysqlMutationMetadata
  • MysqlMutationMapper
    • InsertMutationMapper
      • UpdateMutationMapper
        • DeleteMutationMapper
        • 小结
        • doc
        相关产品与服务
        云数据库 SQL Server
        腾讯云数据库 SQL Server (TencentDB for SQL Server)是业界最常用的商用数据库之一,对基于 Windows 架构的应用程序具有完美的支持。TencentDB for SQL Server 拥有微软正版授权,可持续为用户提供最新的功能,避免未授权使用软件的风险。具有即开即用、稳定可靠、安全运行、弹性扩缩等特点。
        领券
        问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档
        http://www.vxiaotou.com