侧边栏壁纸
博主头像
Leo

  • 累计撰写 233 篇文章
  • 累计创建 274 个标签
  • 累计收到 0 条评论
标签搜索

目 录CONTENT

文章目录
Dev

Elasticsearch常用操作

Leo
Leo
2018-08-10 / 0 评论 / 0 点赞 / 52 阅读 / 2,874 字 / 正在检测是否收录...

此篇主要讲Elasticsearch的常用操作,包含浏览器、命令行、python的相关操作。

由于项目还没完成,不能很好的总结,先把相关的参考资料贴出来


简介

https://www.elastic.co/cn/products/elasticsearch
Elasticsearch简单操作:https://mp.weixin.qq.com/s/1DZuhOvJZNYtbRxYj05pzQ


专题

大数据搜索与可视化分析专题(elasticsearch)
手把手教你使用Flask搭建ES搜索引擎(实战篇):https://mp.weixin.qq.com/s/e3PvJ7_omzUcpHq951qAHA


Elasticsearch

如果使用类似PostMan之类的软件可以直接在Authorization选项卡填写账户和密码访问,如果要模拟浏览器,js等访问,在Headers中带参数,key:Authorization volume:[用浏览器登录,找到Authorization: Basic ZWxhc3RpYyUzQTEyMzQ1Ng==,其中Basic ZWxhc3RpYyUzQTEyMzQ1Ng==就是账户密码的base64编码。参考:安装部署Elasticsearch并用postman浏览数据

Github:https://github.com/elastic/elasticsearch
文档:https://www.elastic.co/guide/cn/elasticsearch/guide/cn/index.html
https://es.xiaoleilu.com/010_Intro/00_README.html

常规操作

Elasticsearch权威指南(中文版)
http://wiki.jikexueyuan.com/project/elasticsearch-definitive-guide-cn/
https://es.xiaoleilu.com/010_Intro/00_README.html

# 常用操作
查看所有数据
http://localhost:9200/_search
# 加?pretty=true是返回方便阅读的json数据
http://localhost:9200/_search?pretty=true

#可查看的东西,类似帮助:
http://127.0.0.1:9200/_cat/

#查看集群的健康状态。
http://127.0.0.1:9200/_cat/health?v

#查看集群的索引数。
http://127.0.0.1:9200/_cat/indices?v

#查看集群所在磁盘的分配状况
http://127.0.0.1:9200/_cat/allocation?v

#查看集群的节点
http://127.0.0.1:9200/_cat/nodes?v

# 查看所有Index
http://localhost:9200/_cat/indices?v

# 列出每个 Index 所包含的 Type(查看数据结构)
http://localhost:9200/_mapping?pretty=true

参考:http://www.ruanyifeng.com/blog/2017/08/elasticsearch.html
常见问题:https://blog.csdn.net/qq_21387171/article/details/53577115
http://www.bubuko.com/infodetail-1989234.html
批量导入数据到elasticsearch中:https://blog.csdn.net/xiaoxinwenziyao/article/details/49471977


高级操作

倒排索引、批量操作

Elasticsearch学习随笔与Scrapy中Elasticsearch的应用:https://mp.weixin.qq.com/s/LcalB_AU9ysmMlv_wXO7Ow


根据地理位置一定范围内的点、排序

注意:基于ElasticSearch 6.2.4,理论上6.X都可以使用。

使用PostMan测试(导入json文件建立所以、初始化数据):

{
	"id": "83822e84-1688-40fd-ad00-dcef96701b63",
	"name": "ES地理距离查询",
	"description": null,
	"auth": null,
	"events": null,
	"variables": null,
	"order": [],
	"folders_order": [
		"0a0eb863-69c3-4c40-a2cc-2afd51b3faed"
	],
	"folders": [
		{
			"id": "0a0eb863-69c3-4c40-a2cc-2afd51b3faed",
			"name": "elasticsearch直接操作",
			"description": null,
			"auth": null,
			"events": null,
			"collection": "83822e84-1688-40fd-ad00-dcef96701b63",
			"folder": null,
			"order": [
				"f71f0ff8-ed4e-4ec0-a909-d49d913c985a",
				"714a3a4c-5ea5-4f11-951f-e97c849bffe3",
				"fd96e4d0-4c68-4caf-9684-506c644edc92"
			],
			"folders_order": [
				"049701e8-8bbb-4a41-9a38-71821e8ecd48"
			]
		},
		{
			"id": "049701e8-8bbb-4a41-9a38-71821e8ecd48",
			"name": "地理位置的索引",
			"description": null,
			"auth": null,
			"events": null,
			"collection": "83822e84-1688-40fd-ad00-dcef96701b63",
			"folder": "0a0eb863-69c3-4c40-a2cc-2afd51b3faed",
			"order": [
				"aafc51a8-b1ed-479e-ad15-edfd4914b8c0",
				"38142d20-e5f4-4cd5-863f-26604418800c",
				"4b80b0de-849a-4887-9183-c6ec6a087cf3",
				"18b991bc-2d2a-49ed-9562-934bd630d495",
				"6ed5ae69-1cc2-465f-8bd6-13378576a7c9",
				"3663c7d9-6487-4d9e-b233-2aa48e169856",
				"aba86b39-e1f6-4bf2-bb0d-c6057e85da37"
			],
			"folders_order": []
		}
	],
	"requests": [
		{
			"id": "18b991bc-2d2a-49ed-9562-934bd630d495",
			"name": "创建索引文档 2",
			"url": "http://localhost:9200/show/test",
			"description": null,
			"data": [],
			"dataMode": "raw",
			"headerData": [
				{
					"key": "Content-Type",
					"value": "application/json"
				}
			],
			"method": "POST",
			"pathVariableData": [],
			"queryParams": [],
			"auth": null,
			"events": null,
			"folder": "049701e8-8bbb-4a41-9a38-71821e8ecd48",
			"rawModeData": "{ \n    \"id\" : 2, \n    \"username\" :  \"深圳市第三人民医院\", \n    \"description\" :  \"现在想要修改为string类型\" ,\n    \"position\":{\n    \t\"location\" : {\n            \"lat\" : 22.6352587415,\n            \"lon\" : 114.1289020619\n        }\n    }\n}",
			"headers": "Content-Type: application/json\n",
			"pathVariables": {}
		},
		{
			"id": "3663c7d9-6487-4d9e-b233-2aa48e169856",
			"name": "创建索引文档 4",
			"url": "http://localhost:9200/show/test",
			"description": null,
			"data": [],
			"dataMode": "raw",
			"headerData": [
				{
					"key": "Content-Type",
					"value": "application/json"
				}
			],
			"method": "POST",
			"pathVariableData": [],
			"queryParams": [],
			"auth": null,
			"events": null,
			"folder": "049701e8-8bbb-4a41-9a38-71821e8ecd48",
			"rawModeData": "{ \n    \"id\" : 3, \n    \"username\" :  \"深圳北站\", \n    \"description\" :  \"现在想要修改为string类型\" ,\n    \"position\":{\n    \t\"location\" : {\n            \"lat\" : 22.6083194998,\n            \"lon\" : 114.0300216169\n        }\n    }\n}",
			"headers": "Content-Type: application/json\n",
			"pathVariables": {}
		},
		{
			"id": "38142d20-e5f4-4cd5-863f-26604418800c",
			"name": "创建索引字段类型",
			"url": "http://localhost:9200/show/test/_mapping",
			"description": null,
			"data": [],
			"dataMode": "raw",
			"headerData": [
				{
					"key": "Content-Type",
					"value": "application/json"
				}
			],
			"method": "POST",
			"pathVariableData": [],
			"queryParams": [],
			"auth": null,
			"events": null,
			"folder": "049701e8-8bbb-4a41-9a38-71821e8ecd48",
			"rawModeData": "{ \n    \"test\": { \n            \"_all\":{ \n              \"enabled\":false \n            }, \n            \"properties\": { \n                \"id\": { \n                    \"type\": \"integer\" \n                }, \n                \"username\": { \n                    \"type\": \"text\", \n                    \"analyzer\": \"ik_pinyin_analyzer\" \n                }, \n                \"description\": { \n                    \"type\": \"text\", \n                    \"analyzer\": \"ik_pinyin_analyzer\" \n                },\n                \"position\": {\n\t\t\t        \"properties\": {\n\t\t\t            \"location\": {\n\t\t\t                \"type\": \"geo_point\"\n\t\t\t            }\n\t\t\t        }\n\t\t      }\n            } \n        } \n  }",
			"headers": "Content-Type: application/json\n",
			"pathVariables": {}
		},
		{
			"id": "4b80b0de-849a-4887-9183-c6ec6a087cf3",
			"name": "创建索引文档",
			"url": "http://localhost:9200/show/test",
			"description": null,
			"data": [],
			"dataMode": "raw",
			"headerData": [
				{
					"key": "Content-Type",
					"value": "application/json"
				}
			],
			"method": "POST",
			"pathVariableData": [],
			"queryParams": [],
			"auth": null,
			"events": null,
			"folder": "049701e8-8bbb-4a41-9a38-71821e8ecd48",
			"rawModeData": "{ \n    \"id\" : 1, \n    \"username\" :  \"XXXX大厦\", \n    \"description\" :  \"如果要修改一个字段的类型\",\n    \"position\":{\n    \t\"location\" : {\n            \"lat\" : 22.6482057076,\n            \"lon\" : 114.1250142233\n        }\n    }\n}",
			"headers": "Content-Type: application/json\n",
			"pathVariables": {}
		},
		{
			"id": "6ed5ae69-1cc2-465f-8bd6-13378576a7c9",
			"name": "创建索引文档 3",
			"url": "http://localhost:9200/show/test",
			"description": null,
			"data": [],
			"dataMode": "raw",
			"headerData": [
				{
					"key": "Content-Type",
					"value": "application/json"
				}
			],
			"method": "POST",
			"pathVariableData": [],
			"queryParams": [],
			"auth": null,
			"events": null,
			"folder": "049701e8-8bbb-4a41-9a38-71821e8ecd48",
			"rawModeData": "{ \n    \"id\" : 3, \n    \"username\" :  \"深圳百合医院\", \n    \"description\" :  \"现在想要修改为string类型\" ,\n    \"position\":{\n    \t\"location\" : {\n            \"lat\" : 22.6164455768,\n            \"lon\" : 114.1395956293\n        }\n    }\n}",
			"headers": "Content-Type: application/json\n",
			"pathVariables": {}
		},
		{
			"id": "714a3a4c-5ea5-4f11-951f-e97c849bffe3",
			"name": "查看所有索引",
			"url": "http://localhost:9200/show/test/_search",
			"description": null,
			"data": [],
			"dataMode": null,
			"headerData": [],
			"method": "GET",
			"pathVariableData": [],
			"queryParams": [],
			"auth": null,
			"events": null,
			"folder": "0a0eb863-69c3-4c40-a2cc-2afd51b3faed",
			"headers": "",
			"pathVariables": {}
		},
		{
			"id": "aafc51a8-b1ed-479e-ad15-edfd4914b8c0",
			"name": "创建索引",
			"url": "http://localhost:9200/show",
			"description": null,
			"data": [],
			"dataMode": "raw",
			"headerData": [
				{
					"key": "Content-Type",
					"value": "application/json"
				}
			],
			"method": "PUT",
			"pathVariableData": [],
			"queryParams": [],
			"auth": null,
			"events": null,
			"folder": "049701e8-8bbb-4a41-9a38-71821e8ecd48",
			"rawModeData": "{\n \"index\": { \n        \"analysis\": { \n            \"analyzer\": { \n                \"ik_pinyin_analyzer\": { \n                    \"type\": \"custom\", \n                    \"tokenizer\": \"ik_max_word\", \n                    \"filter\": [\"my_pinyin\", \"word_delimiter\"] \n                } \n            }, \n            \"filter\": { \n                \"my_pinyin\": { \n                    \"type\": \"pinyin\" \n                } \n            } \n        } \n    }\t\n}",
			"headers": "Content-Type: application/json\n",
			"pathVariables": {}
		},
		{
			"id": "aba86b39-e1f6-4bf2-bb0d-c6057e85da37",
			"name": "根据地理位置查询索引",
			"url": "http://localhost:9200/show/test/_search",
			"description": null,
			"data": [],
			"dataMode": "raw",
			"headerData": [
				{
					"key": "Content-Type",
					"value": "application/json"
				}
			],
			"method": "POST",
			"pathVariableData": [],
			"queryParams": [],
			"auth": null,
			"events": null,
			"folder": "049701e8-8bbb-4a41-9a38-71821e8ecd48",
			"rawModeData": "{\n    \"query\": {\n        \"bool\": {\n        \t\"must\": {\n        \t\t\"match_all\": {}\t\n        \t},\n            \"filter\": {\n                \"geo_distance\": {\n                \t\"distance\" : \"10km\",\n                \t\"position.location\": {\n\t                    \"lat\": 22.6497899384,\n\t                    \"lon\": 114.1258725301\n                \t}\n                }\n            }\n        }\n    }\n}",
			"headers": "Content-Type: application/json\n",
			"pathVariables": {}
		},
		{
			"id": "f71f0ff8-ed4e-4ec0-a909-d49d913c985a",
			"name": "删除索引show",
			"url": "http://localhost:9200/show",
			"description": null,
			"data": [],
			"dataMode": null,
			"headerData": [],
			"method": "DELETE",
			"pathVariableData": [],
			"queryParams": [],
			"auth": null,
			"events": null,
			"folder": "0a0eb863-69c3-4c40-a2cc-2afd51b3faed",
			"headers": "",
			"pathVariables": {}
		},
		{
			"id": "fd96e4d0-4c68-4caf-9684-506c644edc92",
			"name": "根据分词查询索引",
			"url": "http://localhost:9200/show/store/_search",
			"description": null,
			"data": [],
			"dataMode": null,
			"headerData": [],
			"method": "GET",
			"pathVariableData": [],
			"queryParams": [],
			"auth": null,
			"events": null,
			"folder": "0a0eb863-69c3-4c40-a2cc-2afd51b3faed",
			"headers": "",
			"pathVariables": {}
		}
	]
}

下面是查询

POST /[index]/[Type]/_search

# 查询语句
{
    "query": {
        "bool": {
        	"must": {
        		"match_all": {}	
        	},
            "filter": {
                "geo_distance": {
                	"distance" : "1km",
                	"position.location": {
	                    "lat": 22.6497899384,  /* 经度 */
	                    "lon": 114.1258725301  /* 纬度 */
                	}
                }
            }
        }
    }
}

# 返回结果
{
    "took": 3,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 1,
        "hits": [
            {
                "_index": "show",
                "_type": "test",
                "_id": "AZmrv2oBOW57k1nJAlbS",
                "_score": 1,
                "_source": {
                    "id": 1,
                    "username": "XXXX大厦",
                    "description": "XXX大厦,坐落于XXX",
                    "position": {
                        "location": {
                            "lat": 22.6482057076,
                            "lon": 114.1250142233
                        }
                    }
                }
            }
        ]
    }
}

###########################################
# 按距离最小排序,查询语句
{
    "from": 0,   /* 从第0页开始 */
    "size": 10,  /* 返回10条数据 */
    "query": {

        "bool": {
            "must": {
                "match_all": {}
            },
            "filter": {
                "geo_distance": {
                    "distance": "10km",
                    "position.location": {
                        "lat": 22.6497899384,  /* 经度 */
                        "lon": 114.1258725301  /* 纬度 */
                    }
                }
            }
        }
    },
    "sort": [
        {
            "_geo_distance": {
                "position.location": {
                    "lat": 22.6497899384,  /* 经度 */
                    "lon": 114.1258725301  /* 纬度 */
                },
                "order": "asc",  /* 正序asc,倒序desc */
                "unit": "m",    /* 结果排序单位,可以自主调节 */
                "mode": "min"  /* 最小排序 */
            }
    }
    ]
}

# 排序返回结果
{
    "took": 3,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 3,
        "max_score": null,
        "hits": [
            {
                "_index": "show",
                "_type": "test",
                "_id": "AZmrv2oBOW57k1nJAlbS",
                "_score": null,
                "_source": {
                    "id": 1,
                    "username": "XXXX大厦",
                    "description": "XXX大厦,坐落于XXX",
                    "position": {
                        "location": {
                            "lat": 22.6482057076,
                            "lon": 114.1250142233
                        }
                    }
                },
                "sort": [
                    196.95848610299512   /* 距离指定的坐标点196米 */
                ]
            }
        ]
    }
}

参考资料:
http://cwiki.apachecn.org/pages/viewpage.action?pageId=10029641
https://doc.yonyoucloud.com/doc/mastering-elasticsearch/chapter-2/28_README.html
http://cn.voidcc.com/question/p-duyrmmqs-my.html



Elasticsearch DSL

Elasticsearch DSL是对Elasticsearch接口的进一步封装。

Github:https://github.com/elastic/elasticsearch-dsl-py
文档:https://elasticsearch-dsl.readthedocs.io/en/latest



python的elasticsearch库

https://www.cnblogs.com/shaosks/p/7592229.html

0

评论区