广告系统设计与实现(八) -广告检索系统的设计与实现

8.4 广告检索服务

媒体方发起广告请求，检索服务检索广告数据（条件匹配过程），返回响应

媒体方的请求包含的三个要素
媒体方的请求标识 mediaId
请求基本信息 RequestInfo: requestId,adSlots, App,Geo,Device
匹配信息 FeatureInfo:KeywordFeature, DistrictFeature,ItFeature, FeatureRelation

8.4.1 媒体方请求对象的定义

定义检索服务中媒体方发起的请求对象

@Data
@NoArgsConstructor
@AllArgsConstructor
public class SearchRequest {// 媒体方的请求标识private String mediaId;// 请求基本信息(广告位信息AdSlot、终端信息App、设备信息Device、地域信息Geo)private RequestInfo requestInfo;// 匹配信息(关键字、兴趣、地域)private FeatureInfo featureInfo;/*** 请求信息*/@Data@NoArgsConstructor@AllArgsConstructorpublic static class RequestInfo {//唯一请求IDprivate String requestId;//基本请求信息private List<AdSlot> adSlots;private App app;private Geo geo;private Device device;}/*** 匹配信息*/@Data@NoArgsConstructor@AllArgsConstructorpublic static class FeatureInfo {//根据关键字、地域、兴趣筛选推广单元private KeywordFeature keywordFeature;private DistrictFeature districtFeature;private ItFeature itFeature;//默认全部匹配private FeatureRelation relation = FeatureRelation.AND;}
}

广告位信息AdSlot

@Data
@NoArgsConstructor
@AllArgsConstructor
public class AdSlot {// 广告位编码private String adSlotCode;// 广告位置类型private Integer positionType;// 宽和高private Integer width;private Integer height;// 广告物料类型: 图片, 视频private List<Integer> type;// 最低出价private Integer minCpm;
}

应用终端信息App

@Data
@NoArgsConstructor
@AllArgsConstructor
public class App {// 应用编码private String appCode;// 应用名称private String appName;// 应用包名private String packageName;// 请求页面 activity名称private String activityName;
}

设备信息Device

@Data
@NoArgsConstructor
@AllArgsConstructor
public class Device {// 设备 idprivate String deviceCode;// mac地址private String mac;// 设备ipprivate String ip;// 机型编码private String model;// 分辨率尺寸private String displaySize;// 屏幕尺寸private String screenSize;// 设备序列号private String serialName;
}

地域信息Geo

@Data
@NoArgsConstructor
@AllArgsConstructor
public class Geo {//纬度private Float latitude;//经度private Float longitude;//所在城市private String city;private String province;
}

8.4.2 检索服务响应对象的定义

检索服务中检索系统，根据媒体请求，返回响应

定义响应对象格式

匹配信息 FeatureInfo:KeywordFeature, DistrictFeature,ItFeature, FeatureRelation

@Data
@NoArgsConstructor
@AllArgsConstructor
public class SearchResponse {public Map<String, List<Creative>> adSlot2Ads = new HashMap<>();/*** 返回创意信息*/@Data@NoArgsConstructor@AllArgsConstructorpublic static class Creative {private Long adId;private String adUrl;private Integer width;private Integer height;private Integer type;private Integer materialType;// 展示监测 url：监测上架需要展示的广告private List<String> showMonitorUrl = Arrays.asList("www.imooc.com", "www.imooc.com");// 点击监测 url：监测需要点击的广告，类似手机开屏广告private List<String> clickMonitorUrl = Arrays.asList("www.imooc.com", "www.imooc.com");}//广告创意索引对象CreativeObject -> 响应返回创意信息Creativepublic static Creative convert(CreativeObject object) {Creative creative = new Creative();creative.setAdId(object.getAdId());creative.setAdUrl(object.getAdUrl());creative.setWidth(object.getWidth());creative.setHeight(object.getHeight());creative.setType(object.getType());creative.setMaterialType(object.getMaterialType());return creative;}
}

广告二次筛选：地域DistrictFeature

@Data
@NoArgsConstructor
@AllArgsConstructor
public class DistrictFeature {private List<ProvinceAndCity> districts;@Data@NoArgsConstructor@AllArgsConstructorpublic static class ProvinceAndCity {private String province;private String city;}
}

广告二次筛选：兴趣ItFeature

@Data
@NoArgsConstructor
@AllArgsConstructor
public class ItFeature {private List<String> its;
}

广告二次筛选：关键字KeywordFeature

@Data
@NoArgsConstructor
@AllArgsConstructor
public class KeywordFeature {private List<String> keywords;
}

广告筛选关系：全部匹配and，部分匹配or

public enum FeatureRelation {OR,AND
}

广告单元对象的位置类型字段信息

public class AdUnitConstants {public static class POSITION_TYPE {//开屏（打卡应用）public static final int KAIPING = 1;//贴片（视频播放前）public static final int TIEPIAN = 2;//中贴（视频播放中间）public static final int TIEPIAN_MIDDLE = 4;//暂停贴（暂停时广告）public static final int TIEPIAN_PAUSE = 8;//后贴（视频播放完）public static final int TIEPIAN_POST = 16;}
}

在AdUnit索引对象AdUnitObject中添加isAdSlotTypeOK方法，判断检索请求对象adSlotType 和广告单元索引AdUnit ，两者的positionType是否匹配

public static boolean isAdSlotTypeOK(int adSlotType, int positionType) {switch (adSlotType) {case AdUnitConstants.POSITION_TYPE.KAIPING:return isKaiPing(positionType);case AdUnitConstants.POSITION_TYPE.TIEPIAN:return isTiePian(positionType);case AdUnitConstants.POSITION_TYPE.TIEPIAN_MIDDLE:return isTiePianMiddle(positionType);case AdUnitConstants.POSITION_TYPE.TIEPIAN_PAUSE:return isTiePianPause(positionType);case AdUnitConstants.POSITION_TYPE.TIEPIAN_POST:return isTiePianPost(positionType);default:return false;}
}/*** 判断广告位置信息positionType是否匹配预定类型*/
private static boolean isKaiPing(int positionType) {return (positionType & AdUnitConstants.POSITION_TYPE.KAIPING) > 0;
}
private static boolean isTiePian(int positionType) {return (positionType & AdUnitConstants.POSITION_TYPE.TIEPIAN) > 0;
}
private static boolean isTiePianMiddle(int positionType) {return (positionType & AdUnitConstants.POSITION_TYPE.TIEPIAN_MIDDLE) > 0;
}
private static boolean isTiePianPause(int positionType) {return (positionType & AdUnitConstants.POSITION_TYPE.TIEPIAN_PAUSE) > 0;
}
private static boolean isTiePianPost(int positionType) {return (positionType & AdUnitConstants.POSITION_TYPE.TIEPIAN_POST) > 0;
}

在AdUnitIndex中匹配推广单元中位置信息positionType

private static Map<Long, AdUnitObject> objectMap;
static {objectMap = new ConcurrentHashMap<>();}
//匹配推广单元中位置信息positionType
public Set<Long> match(Integer positionType) {Set<Long> adUnitIds = new HashSet<>();//遍历所有索引对象，添加匹配positionType成功的推广单元objectMap.forEach((k, v) -> {if (AdUnitObject.isAdSlotTypeOK(positionType, v.getPositionType())) {adUnitIds.add(k);}});return adUnitIds;
}

8.4.3 构造检索服务的响应对象

获取响应接口

public interface ISearch {//获取广告创意数据SearchResponse fetchAds(SearchRequest request);
}

检索服务的匹配过程
核心的思想是循环遍历媒体方请求的广告位，将匹配范围由大变小，越是能过滤更多的推广单元的条件匹配，越是先执行。对于每一个广告位，匹配过程如下：
构造检索服务的响应对象，根据广告位置类型实现对推广单元的预筛选，根据匹配信息实现对推广单元的再筛选

通过推广单元获取关联的创意实现，填充检索服务响应对象

@Slf4j
@Service
public class SearchImpl implements ISearch {public SearchResponse fallback(SearchRequest request, Throwable e) {return null;}@Override@HystrixCommand(fallbackMethod = "fallback")public SearchResponse fetchAds(SearchRequest request) {// 请求的广告位信息List<AdSlot> adSlots = request.getRequestInfo().getAdSlots();// 响应对象FeatureKeywordFeature keywordFeature = request.getFeatureInfo().getKeywordFeature();DistrictFeature districtFeature = request.getFeatureInfo().getDistrictFeature();ItFeature itFeature = request.getFeatureInfo().getItFeature();FeatureRelation relation = request.getFeatureInfo().getRelation();// 构造响应对象SearchResponse response = new SearchResponse();Map<String, List<SearchResponse.Creative>> adSlot2Ads = response.getAdSlot2Ads();//遍历AdSlotfor (AdSlot adSlot : adSlots) {Set<Long> targetUnitIdSet;//根据广告位置类型一次筛选，获取初始推广单元AdUnitSet<Long> adUnitIdSet = DataTable.of(AdUnitIndex.class).match(adSlot.getPositionType());//根据（地域/兴趣/关键字）特征信息进行and/or二次筛选if (relation == FeatureRelation.AND) {filterKeywordFeature(adUnitIdSet, keywordFeature);filterDistrictFeature(adUnitIdSet, districtFeature);filterItTagFeature(adUnitIdSet, itFeature);targetUnitIdSet = adUnitIdSet;} else {targetUnitIdSet = getORRelationUnitIds(adUnitIdSet, keywordFeature, districtFeature, itFeature);}//根据广告单元ID集合获取广告单元索引对象List<AdUnitObject> unitObjects = DataTable.of(AdUnitIndex.class).fetch(targetUnitIdSet);//判断广告单元状态，筛选无效的广告单元filterAdUnitAndPlanStatus(unitObjects, CommonStatus.VALID);//根据广告单元ID集合获取创意ID集合List<Long> adIds = DataTable.of(CreativeUnitIndex.class).selectAds(unitObjects);//根据创意ID集合获取创意对象集合List<CreativeObject> creatives = DataTable.of(CreativeIndex.class).fetch(adIds);//根据广告位信息AdSlot 实现对创意对象的过滤filterCreativeByAdSlot(creatives, adSlot.getWidth(), adSlot.getHeight(), adSlot.getType());adSlot2Ads.put(adSlot.getAdSlotCode(), buildCreativeResponse(creatives));}log.info("fetchAds: {}-{}", JSON.toJSONString(request), JSON.toJSONString(response));return response;}/*** 广告筛选关系为部分匹配*/private Set<Long> getORRelationUnitIds(Set<Long> adUnitIdSet, KeywordFeature keywordFeature,DistrictFeature districtFeature, ItFeature itFeature) {//判空if (CollectionUtils.isEmpty(adUnitIdSet)) { return Collections.emptySet(); }//将广告单元集合保存三个副本Set<Long> keywordUnitIdSet = new HashSet<>(adUnitIdSet);Set<Long> districtUnitIdSet = new HashSet<>(adUnitIdSet);Set<Long> itUnitIdSet = new HashSet<>(adUnitIdSet);//三个副本按照每个条件进行一次过滤filterKeywordFeature(keywordUnitIdSet, keywordFeature);filterDistrictFeature(districtUnitIdSet, districtFeature);filterItTagFeature(itUnitIdSet, itFeature);//返回三次过滤结果的并集合return new HashSet<>(CollectionUtils.union(CollectionUtils.union(keywordUnitIdSet, districtUnitIdSet), itUnitIdSet));}/*** 根据关键字二次筛选广告单元*/private void filterKeywordFeature(Collection<Long> adUnitIds, KeywordFeature keywordFeature) {//判空if (CollectionUtils.isEmpty(adUnitIds)) { return; }//遍历adUnitIds，调用UnitKeywordIndex中的match方法，判断adUnit中是否含该关键字if (CollectionUtils.isNotEmpty(keywordFeature.getKeywords())) {CollectionUtils.filter(adUnitIds, adUnitId ->DataTable.of(UnitKeywordIndex.class).match(adUnitId, keywordFeature.getKeywords()));}}/*** 根据地域二次筛选广告单元*/private void filterDistrictFeature(Collection<Long> adUnitIds, DistrictFeature districtFeature) {//判空if (CollectionUtils.isEmpty(adUnitIds)) { return; }//遍历adUnitIds，调用UnitDistrictIndex中的match方法，判断adUnit中是否含该地域if (CollectionUtils.isNotEmpty(districtFeature.getDistricts())) {CollectionUtils.filter(adUnitIds, adUnitId ->DataTable.of(UnitDistrictIndex.class).match(adUnitId, districtFeature.getDistricts()));}}/*** 根据兴趣二次筛选广告单元*/private void filterItTagFeature(Collection<Long> adUnitIds, ItFeature itFeature) {//判空if (CollectionUtils.isEmpty(adUnitIds)) { return; }//遍历adUnitIds，调用UnitDistrictIndex中的match方法，判断adUnit中是否含该地域if (CollectionUtils.isNotEmpty(itFeature.getIts())) {CollectionUtils.filter(adUnitIds, adUnitId ->DataTable.of(UnitItIndex.class).match(adUnitId, itFeature.getIts()));}}/*** 判断广告单元状态，筛选无效的广告单元*/private void filterAdUnitAndPlanStatus(List<AdUnitObject> unitObjects, CommonStatus status) {//判空if (CollectionUtils.isEmpty(unitObjects)) { return; }CollectionUtils.filter(unitObjects, object -> object.getUnitStatus().equals(status.getStatus())&& object.getAdPlanObject().getPlanStatus().equals(status.getStatus()));}/*** 通过广告位信息AdSlot 实现对创意对象的过滤*/private void filterCreativeByAdSlot(List<CreativeObject> creatives, Integer width,Integer height, List<Integer> type) {//判空if (CollectionUtils.isEmpty(creatives)) { return; }//筛选状态有效，符合广告位定义的宽高和类型的创意对象CollectionUtils.filter(creatives, creative ->creative.getAuditStatus().equals(CommonStatus.VALID.getStatus())&& creative.getWidth().equals(width)&& creative.getHeight().equals(height)&& type.contains(creative.getType()));}/*** 一个广告位对应一个广告创意* 实现过滤的广告创意对象，转换成检索系统返回响应中的创意信息* CreativeObject -> SearchResponse : Creative*/private List<SearchResponse.Creative> buildCreativeResponse(List<CreativeObject> creatives) {//判空if (CollectionUtils.isEmpty(creatives)) { return Collections.emptyList(); }CreativeObject randomObject = creatives.get(Math.abs(new Random().nextInt()) % creatives.size());return Collections.singletonList(SearchResponse.convert(randomObject));}
}

8.4.5 完善广告检索服务入口

在SearchController中定义服务入口，根据请求，返回响应

@Autowired
private final ISearch search;@PostMapping("/fetchAds")
public SearchResponse fetchAds(@RequestBody SearchRequest request){log.info("ad-search: fetchAds -> {}",JSON.toJSONString(request));return search.fetchAds(request);
}

在ad-gateway网关中resources/application.yml定义

#检索系统
search:path: /ad-search/**serviceId: eureka-client-ad-searchstrip-prefix: false