《MongoDB极简教程》第一章 NoSQL简史 & MongoDB安装&环境配置

本文涉及的产品
云数据库 MongoDB,通用型 2核4GB
简介: MongoDB 是一款开源的文档数据库,并且是业内领先的 NoSQL 数据库,用 C++ 编写而成。NoSQL(NoSQL = Not Only SQL ),意即"不仅仅是SQL"。

MongoDB 是一款开源的文档数据库,并且是业内领先的 NoSQL 数据库,用 C++ 编写而成。

NoSQL

(NoSQL = Not Only SQL ),意即"不仅仅是SQL"。

在现代的计算系统上每天网络上都会产生庞大的数据量。
这些数据有很大一部分是由关系数据库管理系统(RDMBSs)来处理。 1970年 E.F.Codd's提出的关系模型的论文 "A relational model of data for large shared data banks",这使得数据建模和应用程序编程更加简单。

通过应用实践证明,关系模型是非常适合于客户服务器编程,远远超出预期的利益,今天它是结构化数据存储在网络和商务应用的主导技术。

NoSQL 是一项全新的数据库革命性运动,早期就有人提出,发展至2009年趋势越发高涨。NoSQL的拥护者们提倡运用非关系型的数据存储,相对于铺天盖地的关系型数据库运用,这一概念无疑是一种全新的思维的注入。

RDBMS vs NoSQL
RDBMS

  • 高度组织化结构化数据
  • 结构化查询语言(SQL) (SQL)
  • 数据和关系都存储在单独的表中。
  • 数据操纵语言,数据定义语言
  • 严格的一致性
  • 基础事务
    NoSQL
  • 代表着不仅仅是SQL
  • 没有声明性查询语言
  • 没有预定义的模式
    -键 - 值对存储,列存储,文档存储,图形数据库
  • 最终一致性,而非ACID属性
  • 非结构化和不可预知的数据
  • CAP定理
  • 高性能,高可用性和可伸缩性

NoSQL 简史

NoSQL一词最早出现于1998年,是Carlo Strozzi开发的一个轻量、开源、不提供SQL功能的关系数据库。

2009年,Last.fm的Johan Oskarsson发起了一次关于分布式开源数据库的讨论[2],来自Rackspace的Eric Evans再次提出了NoSQL的概念,这时的NoSQL主要指非关系型、分布式、不提供ACID的数据库设计模式。

2009年在亚特兰大举行的"no:sql(east)"讨论会是一个里程碑,其口号是"select fun, profit from real_world where relational=false;"。因此,对NoSQL最普遍的解释是"非关联型的",强调Key-Value Stores和文档数据库的优点,而不是单纯的反对RDBMS。

CAP定理(CAP theorem)

在计算机科学中, CAP定理(CAP theorem), 又被称作 布鲁尔定理(Brewer's theorem), 它指出对于一个分布式计算系统来说,不可能同时满足以下三点:

一致性(Consistency) (所有节点在同一时间具有相同的数据)
可用性(Availability) (保证每个请求不管成功或者失败都有响应)
分隔容忍(Partition tolerance) (系统中任意信息的丢失或失败不会影响系统的继续运作)

CAP理论的核心是:一个分布式系统不可能同时很好的满足一致性,可用性和分区容错性这三个需求,最多只能同时较好的满足两个。
因此,根据 CAP 原理将 NoSQL 数据库分成了满足 CA 原则、满足 CP 原则和满足 AP 原则三 大类:

CA - 单点集群,满足一致性,可用性的系统,通常在可扩展性上不太强大。
CP - 满足一致性,分区容忍性的系统,通常性能不是特别高。
AP - 满足可用性,分区容忍性的系统,通常可能对一致性要求低一些。
cap-theoram-image

NoSQL的优点/缺点

优点:

  • 高可扩展性
  • 分布式计算
  • 低成本
  • 架构的灵活性,半结构化数据
  • 没有复杂的关系

缺点:

  • 没有标准化
  • 有限的查询功能(到目前为止)
  • 最终一致是不直观的程序

BASE

BASE:Basically Available, Soft-state, Eventually Consistent。 由 Eric Brewer 定义。

CAP理论的核心是:一个分布式系统不可能同时很好的满足一致性,可用性和分区容错性这三个需求,最多只能同时较好的满足两个。
BASE是NoSQL数据库通常对可用性及一致性的弱要求原则:

Basically Availble --基本可用
Soft-state --软状态/柔性事务。 "Soft state" 可以理解为"无连接"的, 而 "Hard state" 是"面向连接"的
Eventual Consistency --最终一致性 最终一致性, 也是是 ACID 的最终目的。

MongoDB 特性&优势

MongoDB 特性 优势
事务支持 MongoDB 目前只支持单文档事务,需要复杂事务支持的场景暂时不适合
灵活的文档模型 JSON 格式存储最接近真实对象模型,对开发者友好,方便快速开发迭代
高可用复制集 满足数据高可靠、服务高可用的需求,运维简单,故障自动切换
可扩展分片集群 海量数据存储,服务能力水平扩展
高性能 mmapv1、wiredtiger、mongorocks(rocksdb)、in-memory 等多引擎支持满足各种场景需求
强大的索引支持 地理位置索引可用于构建 各种 O2O 应用、文本索引解决搜索的需求、TTL索引解决历史数据自动过期的需求
Gridfs 解决文件存储的需求
aggregation & mapreduce 解决数据分析场景需求,用户可以自己写查询语句或脚本,将请求都分发到 MongoDB 上完成

文档参考

中文社区

MongoDB 极简实践入门

安装&环境配置

下载:https://www.mongodb.com/download-center#community

安装:
https://docs.mongodb.com/manual/administration/install-community/

解压,配置环境 变量PATH即可。

MONGODB_HOME=/Users/jack/soft/mongodb-osx-x86_64-3.4.2
export PATH=$PATH:$MONGODB_HOME/bin

启动脚本

cat start_mongodb.sh 
#!/bin/bash
source ~/.bashrc

mongod --dbpath ~/dbdata

启动服务进程

./start_mongodb.sh 

2017-03-19T02:50:44.491+0800 I CONTROL  [initandlisten] MongoDB starting : pid=67062 port=27017 dbpath=/Users/jack/dbdata 64-bit host=ChenJackdeMacBook-Pro.local
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] db version v3.4.2
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] git version: 3f76e40c105fc223b3e5aac3e20dcd026b83b38b
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] allocator: system
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] modules: none
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] build environment:
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten]     distarch: x86_64
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten]     target_arch: x86_64
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] options: { storage: { dbPath: "/Users/jack/dbdata" } }
2017-03-19T02:50:44.494+0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=3584M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2017-03-19T02:50:45.352+0800 I CONTROL  [initandlisten] 
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] 
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] 
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
2017-03-19T02:50:45.748+0800 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/Users/jack/dbdata/diagnostic.data'
2017-03-19T02:50:46.073+0800 I INDEX    [initandlisten] build index on: admin.system.version properties: { v: 2, key: { version: 1 }, name: "incompatible_with_version_32", ns: "admin.system.version" }
2017-03-19T02:50:46.073+0800 I INDEX    [initandlisten]      building index using bulk method; build may temporarily use up to 500 megabytes of RAM
2017-03-19T02:50:46.123+0800 I INDEX    [initandlisten] build index done.  scanned 0 total records. 0 secs
2017-03-19T02:50:46.124+0800 I COMMAND  [initandlisten] setting featureCompatibilityVersion to 3.4
2017-03-19T02:50:46.124+0800 I NETWORK  [thread1] waiting for connections on port 27017



2017-03-19T02:51:24.533+0800 I NETWORK  [thread1] connection accepted from 127.0.0.1:49370 #1 (1 connection now open)
2017-03-19T02:51:24.534+0800 I NETWORK  [conn1] received client metadata from 127.0.0.1:49370 conn1: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "3.4.2" }, os: { type: "Darwin", name: "Mac OS X", architecture: "x86_64", version: "16.4.0" } }


Terminal连接客户端

$ mongo
MongoDB shell version v3.4.2
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.2
Server has startup warnings: 
2017-03-19T02:50:45.352+0800 I CONTROL  [initandlisten] 
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] 
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] 
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
> 
> 
> show dbs;
admin  0.000GB
local  0.000GB

强大的 MongoDB 管理工具

MongoDB Shell Enhancements

https://github.com/Jason-Chen-2017/mongo-hacker

Installation

npm install -g mongo-hacker

Enhancements

Basic UX

  • Sort document keys by default
  • Highlight querytime if verboseShell is enabled
    • In green if querytime is at or below slowms
    • In red if query time is above slowms
  • Default indent is 2 spaces instead of tab
    • Customizable by setting indent key of config
  • Verbose shell is enabled by default -- to disable: setVerboseShell(false)
  • Disable notfication of "Type 'it' for more"
  • Custom prompt: hostname(process-version)[rs_status:set_name] db>
  • Always pretty print. You can still use default format by appending .ugly() to the end of db statement.
  • Show DBs has aligned columns, is sorted by database name and shows less significant digits (in master for Mongo 2.5/2.6)
  • Nicer sh.status() output (remove lastmod, take up less space, colorize chunk's shard)
  • Colorized query output for console/terminal windows supporting ANSI color codes.


    Colorized Output

Additional shell commands

The MongoDB shell offers various "shell commands" (sometimes referred to as "shell helpers" as well) that make interactive use of the shell much more convenient than proper, Javascript-only scripted use of the shell.

To make interactive use of the MongoDB shell even more convenient, mongo-hacker adds the following shell commands:

  • count collections/count tables: count the number of collections in each of the mongo server's databases - by @pvdb
  • count documents/count docs: count the number of documents in all (non-system) collections in the database - by @pvdb
  • count indexes: list all collections and display the size of all indexes - by @cog-g

Some of these commands have hidden features that can be enabled in the mongo-hacker config, to make the command output even more useful:

  • by changing the count_deltas setting to true in config.js, the count documents command will also print out the change in the number of documents since the last count - by @pvdb

API Additions

Scripting

Get a list of database names: (by @pvdb)

db.getMongo().getDatabaseNames()

(note that this method is similar - functionality-wise and usage-wise - to the existing db.getCollectionNames() API method and allows for advanced, cross-database scripting in the MongoDB shell)

General

Filter for a collection of documents:

db.collection.filter(<criteria>)

One for finding a single document:

db.collection.find({ ... }).one() == db.collection.findOne({ ... })

Select for selecting fields to return (projection):

db.collection.find({ ... }).select({ name: 1 })

Reverse for descending sort by insertion order (default) or arbitrary field:

db.collection.find({ ... }).reverse()
db.collection.find({ ... }).reverse('createDate')

Last for finding last inserted document (default) or document last by given field:

db.collection.find({ ... }).last()
db.collection.find({ ... }).last('createDate')

Update, Replace, Upsert and Remove can be called on a DBQuery Object:

db.collection.find({ ... }).update({ ... })  // multi update
db.collection.find({ ... }).replace({ ... }) // single replacement
db.collection.find({ ... }).upsert({ ... })  // single upsert
db.collection.find({ ... }).remove()         // multi remove

Sort, limit, and skip through multi updates and removes:

db.collection.find({ ... }).limit(7).update({ ... })
db.collection.find({ ... }).sort({ ... }).skip(1).limit(3).update({ ... })
db.collection.find({ ... }).limit(3).remove()

Note: The performance of multi updates involving a skip or limit may be worse than those without those specifications due to there being absolutely no native support for this feature in MongoDB itself. It should be understood by the user of this software that use of this feature (by calling update on a cursor rather than a collection) is advanced and experimental. The option to do this sort of operation is purely additive to the MongoDB experience with MongoHacker and usage of it is in no way required. Furthermore, its inclusion in this enhancement does not effect the operation of updates invoked through collections and, in practice, is insanely useful.

Aggregation Framework

The aggregation framework is now fluent as well. You can use it as currently documented or via the chainable methods.

Calling aggregate without an array of operations or $operations will make it a match.

// matches every document
db.collection.aggregate()
db.collection.aggregate({})

// matches documents where the "a" is equal to 1
db.collection.aggregate({a: 1})

// matches documents where "a" is greater than 7
db.collection.aggregate({a: {$gt: 7}})

Additional methods can then be chained on top of the inital match in order to make more complicated aggregations.

// Match and project
db.collection.aggregate(<querydoc>).project(<projection>)
db.collection.aggregate({a: 1}).project({a: 1, _id: 0})

// Match, group and sort
db.collection.aggregate({<match>}).group({<group>}).sort({<sort>})
db.test.aggregate().group({_id: '$a', 'sum': {'$sum': 1}}).sort({sum: -1})

Data Generation

For easy and simple random data generation you can utilise these methods below. You can use any of these functions in a loop. For example:

// Inserts 20 documents with random data. 
for (i=1; i<21; i++) { 
    db.collection.insert(
            {
             word: randomWord(), 
             number: randomNumber(), 
             date: randomDate() 
            }
    ); 
}
randomWord

You can specify the length of each word, the number of words, and an optional seeded word in a sentence randomly. Use the optional seed parameter for testing text search.

randomWord(length=5, words=1, seed=undefined)

// Inserts a random sentence consisting of 5 letters per word, 5 words in total, 
// with a probability to insert the word 'needle' in the sentence
db.collection.insert( { words: randomWord(5, 5, 'needle') } )

// Inserts a random word consisting of 16 letters
db.collection.insert( { words: randomWord(16) } )
randomNumber

You can specify maximum number to be randomly generated (exclusive)

randomNumber(max=100)

// Inserts a random number in the range of 0 or 1. 
db.collection.insert( { number: randomNumber(2) } )

// Inserts a random number in the range of 0 or 999. 
db.collection.insert( { number: randomNumber(1000) } )

randomDate

You can specify start and end dates range to be randomly generated. (exclusive)

randomDate(start= <2 years ago> , end=Date() )

// Inserts a random date object in the range of 1st January 2016 to 1st February 2016
db.collection.insert( { date: randomDate(ISODate("2016-01-01T00:00:00"), ISODate("2016-02-01T00:00:00")) })

// If today is 19th May 2016 and you specify only the start of the day, 
// this will generate random date object between 00:00:00 to current time.  
db.collection.insert( { date: randomDate(ISODate("2016-05-19T00:00:00")) })

Helpers

General Shell Helpers

  • findCommand('search') list commands that match the search string

Aggregation Framework Helpers -- on collections

  • Group and Count: gcount(group_field, filter)
  • Group and Sum: gsum(group_field, sum_field, filter)
  • Group and Average: gavg(group_field, avg_field, filter)

Run function on some/all databases

runOnDbs(/db_names_regexp/, function(db) {
    // callback is ran for each database which name matches regular expression
    // db is that selected database
});

mongo-express

https://github.com/Jason-Chen-2017/mongo-express

Web-based MongoDB admin interface, written with Node.js and express

终端查看的效果:

jack@jacks-MacBook-Air:~$ mongo
MongoDB shell version: 3.2.4
connecting to: test
Mongo-Hacker 0.0.14
Server has startup warnings: 
2017-03-13T10:31:13.023+0800 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2017-03-13T10:31:13.023+0800 I CONTROL  [initandlisten] 
jacks-MacBook-Air(mongod-3.2.4) test> 
jacks-MacBook-Air(mongod-3.2.4) test> 
jacks-MacBook-Air(mongod-3.2.4) test> db
test
jacks-MacBook-Air(mongod-3.2.4) test> show dbs;
admin      → 0.000GB
local      → 0.000GB
restfiddle → 0.001GB
jacks-MacBook-Air(mongod-3.2.4) test> use restfiddle;
switched to db restfiddle
jacks-MacBook-Air(mongod-3.2.4) restfiddle> show collections;
EntityAuth        → 0.000MB / 0.004MB
activityLog       → 0.001MB / 0.035MB
baseNode          → 0.021MB / 0.043MB
config            → 0.000MB / 0.016MB
conversation      → 0.017MB / 0.035MB
httpRequestHeader → 0.005MB / 0.016MB
oAuth2            → 0.002MB / 0.016MB
project           → 0.002MB / 0.031MB
rfRequest         → 0.017MB / 0.035MB
rfResponse        → 0.036MB / 0.047MB
runnerLog         → 0.000MB / 0.016MB
tag               → 0.000MB / 0.016MB
user              → 0.000MB / 0.035MB
workspace         → 0.001MB / 0.031MB
jacks-MacBook-Air(mongod-3.2.4) restfiddle> l
2017-03-21T14:18:55.202+0800 E QUERY    [thread1] ReferenceError: l is not defined :
@(shell):1:1

jacks-MacBook-Air(mongod-3.2.4) restfiddle> db.runnerLog.find()
{
  "_id": ObjectId("575fad7dba637304fffbae85"),
  "_class": "com.restfiddle.entity.RunnerLog",
  "status": "ACTIVE",
  "version": NumberLong("0")
}
{
  "_id": ObjectId("575fad92ba637304fffbae86"),
  "_class": "com.restfiddle.entity.RunnerLog",
  "status": "ACTIVE",
  "version": NumberLong("0")
}
{
  "_id": ObjectId("575fad9cba637304fffbae87"),
  "_class": "com.restfiddle.entity.RunnerLog",
  "status": "ACTIVE",
  "version": NumberLong("0")
}
Fetched 3 record(s) in 7ms
jacks-MacBook-Air(mongod-3.2.4) restfiddle> 

mongo-express的效果:

螢幕快照 2017-03-21 14.25.03.png
螢幕快照 2017-03-21 14.25.57.png
相关实践学习
MongoDB数据库入门
MongoDB数据库入门实验。
快速掌握 MongoDB 数据库
本课程主要讲解MongoDB数据库的基本知识,包括MongoDB数据库的安装、配置、服务的启动、数据的CRUD操作函数使用、MongoDB索引的使用(唯一索引、地理索引、过期索引、全文索引等)、MapReduce操作实现、用户管理、Java对MongoDB的操作支持(基于2.x驱动与3.x驱动的完全讲解)。 通过学习此课程,读者将具备MongoDB数据库的开发能力,并且能够使用MongoDB进行项目开发。 &nbsp; 相关的阿里云产品:云数据库 MongoDB版 云数据库MongoDB版支持ReplicaSet和Sharding两种部署架构,具备安全审计,时间点备份等多项企业能力。在互联网、物联网、游戏、金融等领域被广泛采用。 云数据库MongoDB版(ApsaraDB for MongoDB)完全兼容MongoDB协议,基于飞天分布式系统和高可靠存储引擎,提供多节点高可用架构、弹性扩容、容灾、备份回滚、性能优化等解决方案。 产品详情: https://www.aliyun.com/product/mongodb
相关文章
|
3天前
|
存储 分布式计算 NoSQL
MongoDB的简介和安装(在服务器上)
MongoDB的简介和安装(在服务器上)
18 0
|
6天前
|
NoSQL MongoDB Redis
Python与NoSQL数据库(MongoDB、Redis等)面试问答
【4月更文挑战第16天】本文探讨了Python与NoSQL数据库(如MongoDB、Redis)在面试中的常见问题,包括连接与操作数据库、错误处理、高级特性和缓存策略。重点介绍了使用`pymongo`和`redis`库进行CRUD操作、异常捕获以及数据一致性管理。通过理解这些问题、易错点及避免策略,并结合代码示例,开发者能在面试中展现其技术实力和实践经验。
128 8
Python与NoSQL数据库(MongoDB、Redis等)面试问答
|
1月前
|
NoSQL MongoDB Docker
docker安装MongoDB
docker安装MongoDB
22 0
|
2月前
|
NoSQL 关系型数据库 MySQL
Windows、Linux、Mac安装数据库(mysql、MongoDB、Redis)#0
不同系统下进行MySQL安装、MongoDB安装、Redis安装【2月更文挑战第5天】
445 5
Windows、Linux、Mac安装数据库(mysql、MongoDB、Redis)#0
|
3月前
|
存储 JSON NoSQL
【MongoDB】<文档型数据库>Windows&Liunx安装MongoDB(无错完整)
【1月更文挑战第26天】【MongoDB】<文档型数据库>Windows&Liunx安装MongoDB(无错完整)
|
3月前
|
存储 NoSQL MongoDB
学习如何使用 Python 连接 MongoDB: PyMongo 安装和基础操作教程
Python 需要一个 MongoDB 驱动程序来访问 MongoDB 数据库。我将使用 MongoDB 驱动程序 PyMongo 建议您使用 PIP 来安装 PyMongo。PIP 很可能已经安装在您的 Python 环境中。将命令行导航到 PIP 的位置,然后键入以下内容:
95 1
|
3月前
|
NoSQL 安全 Linux
百度搜索:蓝易云【CentOS7安装MongoDB教程】
这些是在CentOS 7上安装MongoDB的基本步骤。根据您的需求和具体环境,可能还需要进行其他配置和调整。请确保在进行任何与网络连接和安全相关的操作之前,详细了解您的网络环境和安全需求,并采取适当的安全措施。
133 0
|
4月前
|
NoSQL MongoDB 数据库
MongoDB【部署 03】Windows系统安装mongodb并设置用户名密码(无需安装mongosh)及SpringBoot集成报错 Command failed with error 18
MongoDB【部署 03】Windows系统安装mongodb并设置用户名密码(无需安装mongosh)及SpringBoot集成报错 Command failed with error 18
143 0
|
4月前
|
NoSQL MongoDB Docker
百度搜索:蓝易云【Scrapy框架之Docker安装MongoDB教程。】
现在,你已经成功在Scrapy框架中使用Docker安装并配置了MongoDB。你可以在Scrapy爬虫中使用MongoDB进行数据存储和处理。
220 0
|
3天前
|
NoSQL MongoDB 数据库
MongoDB数据恢复—MongoDB数据库文件被破坏的数据恢复案例
服务器数据恢复环境: 一台Windows Server操作系统服务器,服务器上部署MongoDB数据库。 MongoDB数据库故障&检测: 工作人员在未关闭MongoDB数据库服务的情况下,将数据库文件拷贝到其他分区。拷贝完成后将原MongoDB数据库所在分区进行了格式化操作,然后将数据库文件拷回原分区,重新启动MongoDB服务,服务无法启动。