Release 1.158.1 (#1010)

* Expose environment variable: HOST

* Update changelog

.env

@@ -14,4 +14,3 @@ ACCESS_TOKEN_SALT=<INSERT_RANDOM_STRING>
 ALPHA_VANTAGE_API_KEY=
 DATABASE_URL=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@localhost:5432/${POSTGRES_DB}?sslmode=prefer
 JWT_SECRET_KEY=<INSERT_RANDOM_STRING>
-PORT=3333

CHANGELOG.md

@@ -5,6 +5,56 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 1.158.1 - 12.06.2022
+
+### Added
+
+- Extended the queue jobs view in the admin control panel by a data dialog
+
+### Changed
+
+- Exposed the environment variable `HOST`
+- Decreased the number of attempts of queue jobs from `20` to `10` (fail earlier)
+- Improved the message for data provider errors in the client
+- Changed the label from _Balance_ to _Cash Balance_ in the account dialog
+- Restructured the documentation for self-hosting
+
+## 1.157.0 - 11.06.2022
+
+### Added
+
+- Extended the queue jobs view in the admin control panel by the number of attempts and the status
+
+### Changed
+
+- Migrated the historical market data gathering to the queue design pattern
+- Refreshed the cryptocurrencies list to support more coins by default
+- Increased the historical data chart of the _Fear & Greed Index_ (market mood) to 180 days
+- Upgraded `chart.js` from version `3.7.0` to `3.8.0`
+- Upgraded `envalid` from version `7.2.1` to `7.3.1`
+
+### Fixed
+
+- Reloaded the accounts of a user after creating, editing or deleting one
+- Excluded empty items in the activities filter
+
+## 1.156.0 - 05.06.2022
+
+### Added
+
+- Added the user id to the account page
+- Added a new view with jobs of the queue to the admin control panel
+
+### Changed
+
+- Simplified the features page
+- Restructured the _FIRE_ section
+- Upgraded `@simplewebauthn/browser` and `@simplewebauthn/server` from version `4.1.0` to `5.2.1`
+
+### Fixed
+
+- Fixed the `docker-compose` files to resolve variables correctly
+
 ## 1.155.0 - 29.05.2022
 
 ### Added

README.md

@@ -79,47 +79,50 @@ The backend is based on [NestJS](https://nestjs.com) using [PostgreSQL](https://
 
 The frontend is built with [Angular](https://angular.io) and uses [Angular Material](https://material.angular.io) with utility classes from [Bootstrap](https://getbootstrap.com).
 
-## Run with Docker (self-hosting)
+## Self-hosting
 
-### Prerequisites
+### Run with Docker Compose
 
-- [Docker](https://www.docker.com/products/docker-desktop)
+#### Prerequisites
-- A local copy of this Git repository (clone)
 
-### a. Run environment
+- Basic knowledge of Docker
+- Installation of [Docker](https://www.docker.com/products/docker-desktop)
+- Local copy of this Git repository (clone)
+
+#### a. Run environment
 
 Run the following command to start the Docker images from [Docker Hub](https://hub.docker.com/r/ghostfolio/ghostfolio):
 
 ```bash
-docker-compose -f docker/docker-compose.yml up -d
+docker-compose --env-file ./.env -f docker/docker-compose.yml up -d
 ```
 
-#### Setup Database
+##### Setup Database
 
 Run the following command to setup the database once Ghostfolio is running:
 
 ```bash
-docker-compose -f docker/docker-compose.yml exec ghostfolio yarn database:setup
+docker-compose --env-file ./.env -f docker/docker-compose.yml exec ghostfolio yarn database:setup
 ```
 
-### b. Build and run environment
+#### b. Build and run environment
 
 Run the following commands to build and start the Docker images:
 
 ```bash
-docker-compose -f docker/docker-compose.build.yml build
+docker-compose --env-file ./.env -f docker/docker-compose.build.yml build
-docker-compose -f docker/docker-compose.build.yml up -d
+docker-compose --env-file ./.env -f docker/docker-compose.build.yml up -d
 ```
 
-#### Setup Database
+##### Setup Database
 
 Run the following command to setup the database once Ghostfolio is running:
 
 ```bash
-docker-compose -f docker/docker-compose.build.yml exec ghostfolio yarn database:setup
+docker-compose --env-file ./.env -f docker/docker-compose.build.yml exec ghostfolio yarn database:setup
 ```
 
-### Fetch Historical Data
+#### Fetch Historical Data
 
 Open http://localhost:3333 in your browser and accomplish these steps:
 
@@ -127,13 +130,13 @@ Open http://localhost:3333 in your browser and accomplish these steps:
 1. Go to the _Admin Control Panel_ and click _Gather All Data_ to fetch historical data
 1. Click _Sign out_ and check out the _Live Demo_
 
-### Upgrade Version
+#### Upgrade Version
 
 1. Increase the version of the `ghostfolio/ghostfolio` Docker image in `docker/docker-compose.yml`
-1. Run the following command to start the new Docker image: `docker-compose -f docker/docker-compose.yml up -d`
+1. Run the following command to start the new Docker image: `docker-compose --env-file ./.env -f docker/docker-compose.yml up -d`
-1. Then, run the following command to keep your database schema in sync: `docker-compose -f docker/docker-compose.yml exec ghostfolio yarn database:migrate`
+1. Then, run the following command to keep your database schema in sync: `docker-compose --env-file ./.env -f docker/docker-compose.yml exec ghostfolio yarn database:migrate`
 
-## Run with _Unraid_ (self-hosting)
+### Run with _Unraid_ (unofficial)
 
 Please follow the instructions of the Ghostfolio [Unraid Community App](https://unraid.net/community/apps?q=ghostfolio).
 
@@ -149,7 +152,7 @@ Please follow the instructions of the Ghostfolio [Unraid Community App](https://
 ### Setup
 
 1. Run `yarn install`
-1. Run `docker-compose -f docker/docker-compose.dev.yml up -d` to start [PostgreSQL](https://www.postgresql.org) and [Redis](https://redis.io)
+1. Run `docker-compose --env-file ./.env -f docker/docker-compose.dev.yml up -d` to start [PostgreSQL](https://www.postgresql.org) and [Redis](https://redis.io)
 1. Run `yarn database:setup` to initialize the database schema and populate your database with (example) data
 1. Start the server and the client (see [_Development_](#Development))
 1. Create a new user via _Get Started_ (this first user will get the role `ADMIN`)

apps/api/src/app/admin/admin.controller.ts

@@ -2,8 +2,8 @@ import { DataGatheringService } from '@ghostfolio/api/services/data-gathering.se
 import { MarketDataService } from '@ghostfolio/api/services/market-data.service';
 import { PropertyDto } from '@ghostfolio/api/services/property/property.dto';
 import {
-  DATA_GATHERING_QUEUE,
+  GATHER_ASSET_PROFILE_PROCESS,
-  GATHER_ASSET_PROFILE_PROCESS
+  GATHER_ASSET_PROFILE_PROCESS_OPTIONS
 } from '@ghostfolio/common/config';
 import {
   AdminData,
@@ -12,7 +12,6 @@ import {
 } from '@ghostfolio/common/interfaces';
 import { hasPermission, permissions } from '@ghostfolio/common/permissions';
 import type { RequestWithUser } from '@ghostfolio/common/types';
-import { InjectQueue } from '@nestjs/bull';
 import {
   Body,
   Controller,
@@ -28,7 +27,6 @@ import {
 import { REQUEST } from '@nestjs/core';
 import { AuthGuard } from '@nestjs/passport';
 import { DataSource, MarketData } from '@prisma/client';
-import { Queue } from 'bull';
 import { isDate } from 'date-fns';
 import { StatusCodes, getReasonPhrase } from 'http-status-codes';
 
@@ -39,8 +37,6 @@ import { UpdateMarketDataDto } from './update-market-data.dto';
 export class AdminController {
   public constructor(
     private readonly adminService: AdminService,
-    @InjectQueue(DATA_GATHERING_QUEUE)
-    private readonly dataGatheringQueue: Queue,
     private readonly dataGatheringService: DataGatheringService,
     private readonly marketDataService: MarketDataService,
     @Inject(REQUEST) private readonly request: RequestWithUser
@@ -64,6 +60,24 @@ export class AdminController {
     return this.adminService.get();
   }
 
+  @Post('gather')
+  @UseGuards(AuthGuard('jwt'))
+  public async gather7Days(): Promise<void> {
+    if (
+      !hasPermission(
+        this.request.user.permissions,
+        permissions.accessAdminControl
+      )
+    ) {
+      throw new HttpException(
+        getReasonPhrase(StatusCodes.FORBIDDEN),
+        StatusCodes.FORBIDDEN
+      );
+    }
+
+    this.dataGatheringService.gather7Days();
+  }
+
   @Post('gather/max')
   @UseGuards(AuthGuard('jwt'))
   public async gatherMax(): Promise<void> {
@@ -82,10 +96,14 @@ export class AdminController {
     const uniqueAssets = await this.dataGatheringService.getUniqueAssets();
 
     for (const { dataSource, symbol } of uniqueAssets) {
-      await this.dataGatheringQueue.add(GATHER_ASSET_PROFILE_PROCESS, {
+      await this.dataGatheringService.addJobToQueue(
+        GATHER_ASSET_PROFILE_PROCESS,
+        {
           dataSource,
           symbol
-      });
+        },
+        GATHER_ASSET_PROFILE_PROCESS_OPTIONS
+      );
     }
 
     this.dataGatheringService.gatherMax();
@@ -109,10 +127,14 @@ export class AdminController {
     const uniqueAssets = await this.dataGatheringService.getUniqueAssets();
 
     for (const { dataSource, symbol } of uniqueAssets) {
-      await this.dataGatheringQueue.add(GATHER_ASSET_PROFILE_PROCESS, {
+      await this.dataGatheringService.addJobToQueue(
+        GATHER_ASSET_PROFILE_PROCESS,
+        {
           dataSource,
           symbol
-      });
+        },
+        GATHER_ASSET_PROFILE_PROCESS_OPTIONS
+      );
     }
   }
 
@@ -134,10 +156,14 @@ export class AdminController {
       );
     }
 
-    await this.dataGatheringQueue.add(GATHER_ASSET_PROFILE_PROCESS, {
+    await this.dataGatheringService.addJobToQueue(
+      GATHER_ASSET_PROFILE_PROCESS,
+      {
         dataSource,
         symbol
-    });
+      },
+      GATHER_ASSET_PROFILE_PROCESS_OPTIONS
+    );
   }
 
   @Post('gather/:dataSource/:symbol')

apps/api/src/app/admin/admin.module.ts

@@ -11,6 +11,7 @@ import { Module } from '@nestjs/common';
 
 import { AdminController } from './admin.controller';
 import { AdminService } from './admin.service';
+import { QueueModule } from './queue/queue.module';
 
 @Module({
   imports: [
@@ -21,6 +22,7 @@ import { AdminService } from './admin.service';
     MarketDataModule,
     PrismaModule,
     PropertyModule,
+    QueueModule,
     SubscriptionModule,
     SymbolProfileModule
   ],

apps/api/src/app/admin/admin.service.ts

@@ -42,8 +42,6 @@ export class AdminService {
 
   public async get(): Promise<AdminData> {
     return {
-      dataGatheringProgress:
-        await this.dataGatheringService.getDataGatheringProgress(),
       exchangeRates: this.exchangeRateDataService
         .getCurrencies()
         .filter((currency) => {
@@ -60,7 +58,6 @@ export class AdminService {
             )
           };
         }),
-      lastDataGathering: await this.getLastDataGathering(),
       settings: await this.propertyService.get(),
       transactionCount: await this.prismaService.order.count(),
       userCount: await this.prismaService.user.count(),
@@ -161,30 +158,11 @@ export class AdminService {
 
     if (key === PROPERTY_CURRENCIES) {
       await this.exchangeRateDataService.initialize();
-      await this.dataGatheringService.reset();
     }
 
     return response;
   }
 
-  private async getLastDataGathering() {
-    const lastDataGathering =
-      await this.dataGatheringService.getLastDataGathering();
-
-    if (lastDataGathering) {
-      return lastDataGathering;
-    }
-
-    const dataGatheringInProgress =
-      await this.dataGatheringService.getIsInProgress();
-
-    if (dataGatheringInProgress) {
-      return 'IN_PROGRESS';
-    }
-
-    return undefined;
-  }
-
   private async getUsersWithAnalytics(): Promise<AdminData['users']> {
     const usersWithAnalytics = await this.prismaService.user.findMany({
       orderBy: {

apps/api/src/app/admin/queue/queue.controller.ts

@@ -0,0 +1,87 @@
+import { AdminJobs } from '@ghostfolio/common/interfaces';
+import { hasPermission, permissions } from '@ghostfolio/common/permissions';
+import type { RequestWithUser } from '@ghostfolio/common/types';
+import {
+  Controller,
+  Delete,
+  Get,
+  HttpException,
+  Inject,
+  Param,
+  Query,
+  UseGuards
+} from '@nestjs/common';
+import { REQUEST } from '@nestjs/core';
+import { AuthGuard } from '@nestjs/passport';
+import { JobStatus } from 'bull';
+import { StatusCodes, getReasonPhrase } from 'http-status-codes';
+
+import { QueueService } from './queue.service';
+
+@Controller('admin/queue')
+export class QueueController {
+  public constructor(
+    private readonly queueService: QueueService,
+    @Inject(REQUEST) private readonly request: RequestWithUser
+  ) {}
+
+  @Delete('job')
+  @UseGuards(AuthGuard('jwt'))
+  public async deleteJobs(
+    @Query('status') filterByStatus?: string
+  ): Promise<void> {
+    if (
+      !hasPermission(
+        this.request.user.permissions,
+        permissions.accessAdminControl
+      )
+    ) {
+      throw new HttpException(
+        getReasonPhrase(StatusCodes.FORBIDDEN),
+        StatusCodes.FORBIDDEN
+      );
+    }
+
+    const status = <JobStatus[]>filterByStatus?.split(',') ?? undefined;
+    return this.queueService.deleteJobs({ status });
+  }
+
+  @Get('job')
+  @UseGuards(AuthGuard('jwt'))
+  public async getJobs(
+    @Query('status') filterByStatus?: string
+  ): Promise<AdminJobs> {
+    if (
+      !hasPermission(
+        this.request.user.permissions,
+        permissions.accessAdminControl
+      )
+    ) {
+      throw new HttpException(
+        getReasonPhrase(StatusCodes.FORBIDDEN),
+        StatusCodes.FORBIDDEN
+      );
+    }
+
+    const status = <JobStatus[]>filterByStatus?.split(',') ?? undefined;
+    return this.queueService.getJobs({ status });
+  }
+
+  @Delete('job/:id')
+  @UseGuards(AuthGuard('jwt'))
+  public async deleteJob(@Param('id') id: string): Promise<void> {
+    if (
+      !hasPermission(
+        this.request.user.permissions,
+        permissions.accessAdminControl
+      )
+    ) {
+      throw new HttpException(
+        getReasonPhrase(StatusCodes.FORBIDDEN),
+        StatusCodes.FORBIDDEN
+      );
+    }
+
+    return this.queueService.deleteJob(id);
+  }
+}

apps/api/src/app/admin/queue/queue.module.ts

@@ -0,0 +1,12 @@
+import { DataGatheringModule } from '@ghostfolio/api/services/data-gathering.module';
+import { Module } from '@nestjs/common';
+
+import { QueueController } from './queue.controller';
+import { QueueService } from './queue.service';
+
+@Module({
+  controllers: [QueueController],
+  imports: [DataGatheringModule],
+  providers: [QueueService]
+})
+export class QueueModule {}

apps/api/src/app/admin/queue/queue.service.ts

@@ -0,0 +1,65 @@
+import {
+  DATA_GATHERING_QUEUE,
+  QUEUE_JOB_STATUS_LIST
+} from '@ghostfolio/common/config';
+import { AdminJobs } from '@ghostfolio/common/interfaces';
+import { InjectQueue } from '@nestjs/bull';
+import { Injectable, Logger } from '@nestjs/common';
+import { JobStatus, Queue } from 'bull';
+
+@Injectable()
+export class QueueService {
+  public constructor(
+    @InjectQueue(DATA_GATHERING_QUEUE)
+    private readonly dataGatheringQueue: Queue
+  ) {}
+
+  public async deleteJob(aId: string) {
+    return (await this.dataGatheringQueue.getJob(aId))?.remove();
+  }
+
+  public async deleteJobs({
+    status = QUEUE_JOB_STATUS_LIST
+  }: {
+    status?: JobStatus[];
+  }) {
+    const jobs = await this.dataGatheringQueue.getJobs(status);
+
+    for (const job of jobs) {
+      try {
+        await job.remove();
+      } catch (error) {
+        Logger.warn(error, 'QueueService');
+      }
+    }
+  }
+
+  public async getJobs({
+    limit = 1000,
+    status = QUEUE_JOB_STATUS_LIST
+  }: {
+    limit?: number;
+    status?: JobStatus[];
+  }): Promise<AdminJobs> {
+    const jobs = await this.dataGatheringQueue.getJobs(status);
+
+    const jobsWithState = await Promise.all(
+      jobs.slice(0, limit).map(async (job) => {
+        return {
+          attemptsMade: job.attemptsMade + 1,
+          data: job.data,
+          finishedOn: job.finishedOn,
+          id: job.id,
+          name: job.name,
+          stacktrace: job.stacktrace,
+          state: await job.getState(),
+          timestamp: job.timestamp
+        };
+      })
+    );
+
+    return {
+      jobs: jobsWithState
+    };
+  }
+}

apps/api/src/app/app.controller.ts

@@ -1,26 +1,6 @@
-import { DataGatheringService } from '@ghostfolio/api/services/data-gathering.service';
 import { Controller } from '@nestjs/common';
 
-import { RedisCacheService } from './redis-cache/redis-cache.service';
-
 @Controller()
 export class AppController {
-  public constructor(
+  public constructor() {}
-    private readonly dataGatheringService: DataGatheringService,
-    private readonly redisCacheService: RedisCacheService
-  ) {
-    this.initialize();
-  }
-
-  private async initialize() {
-    this.redisCacheService.reset();
-
-    const isDataGatheringInProgress =
-      await this.dataGatheringService.getIsInProgress();
-
-    if (isDataGatheringInProgress) {
-      // Prepare for automatical data gathering, if hung up in progress state
-      await this.dataGatheringService.reset();
-    }
-  }
 }

apps/api/src/app/cache/cache.controller.ts

@@ -1,25 +1,39 @@
-import { CacheService } from '@ghostfolio/api/app/cache/cache.service';
 import { RedisCacheService } from '@ghostfolio/api/app/redis-cache/redis-cache.service';
+import { hasPermission, permissions } from '@ghostfolio/common/permissions';
 import type { RequestWithUser } from '@ghostfolio/common/types';
-import { Controller, Inject, Post, UseGuards } from '@nestjs/common';
+import {
+  Controller,
+  HttpException,
+  Inject,
+  Post,
+  UseGuards
+} from '@nestjs/common';
 import { REQUEST } from '@nestjs/core';
 import { AuthGuard } from '@nestjs/passport';
+import { StatusCodes, getReasonPhrase } from 'http-status-codes';
 
 @Controller('cache')
 export class CacheController {
   public constructor(
-    private readonly cacheService: CacheService,
     private readonly redisCacheService: RedisCacheService,
     @Inject(REQUEST) private readonly request: RequestWithUser
-  ) {
+  ) {}
-    this.redisCacheService.reset();
-  }
 
   @Post('flush')
   @UseGuards(AuthGuard('jwt'))
   public async flushCache(): Promise<void> {
-    this.redisCacheService.reset();
+    if (
+      !hasPermission(
+        this.request.user.permissions,
+        permissions.accessAdminControl
+      )
+    ) {
+      throw new HttpException(
+        getReasonPhrase(StatusCodes.FORBIDDEN),
+        StatusCodes.FORBIDDEN
+      );
+    }
 
-    return this.cacheService.flush();
+    return this.redisCacheService.reset();
   }
 }

apps/api/src/app/cache/cache.module.ts

@@ -1,4 +1,3 @@
-import { CacheService } from '@ghostfolio/api/app/cache/cache.service';
 import { RedisCacheModule } from '@ghostfolio/api/app/redis-cache/redis-cache.module';
 import { ConfigurationModule } from '@ghostfolio/api/services/configuration.module';
 import { DataGatheringModule } from '@ghostfolio/api/services/data-gathering.module';
@@ -11,7 +10,6 @@ import { Module } from '@nestjs/common';
 import { CacheController } from './cache.controller';
 
 @Module({
-  exports: [CacheService],
   controllers: [CacheController],
   imports: [
     ConfigurationModule,
@@ -21,7 +19,6 @@ import { CacheController } from './cache.controller';
     PrismaModule,
     RedisCacheModule,
     SymbolProfileModule
-  ],
+  ]
-  providers: [CacheService]
 })
 export class CacheModule {}

apps/api/src/app/cache/cache.service.ts

@@ -1,15 +0,0 @@
-import { DataGatheringService } from '@ghostfolio/api/services/data-gathering.service';
-import { Injectable } from '@nestjs/common';
-
-@Injectable()
-export class CacheService {
-  public constructor(
-    private readonly dataGaterhingService: DataGatheringService
-  ) {}
-
-  public async flush(): Promise<void> {
-    await this.dataGaterhingService.reset();
-
-    return;
-  }
-}

apps/api/src/app/info/info.service.ts

@@ -106,7 +106,6 @@ export class InfoService {
       baseCurrency: this.configurationService.get('BASE_CURRENCY'),
       currencies: this.exchangeRateDataService.getCurrencies(),
       demoAuthToken: this.getDemoAuthToken(),
-      lastDataGathering: await this.getLastDataGathering(),
       statistics: await this.getStatistics(),
       subscriptions: await this.getSubscriptions(),
       tags: await this.tagService.get()
@@ -215,13 +214,6 @@ export class InfoService {
     });
   }
 
-  private async getLastDataGathering() {
-    const lastDataGathering =
-      await this.dataGatheringService.getLastDataGathering();
-
-    return lastDataGathering ?? null;
-  }
-
   private async getStatistics() {
     if (!this.configurationService.get('ENABLE_FEATURE_STATISTICS')) {
       return undefined;

apps/api/src/app/order/order.service.ts

@@ -1,16 +1,14 @@
 import { AccountService } from '@ghostfolio/api/app/account/account.service';
-import { CacheService } from '@ghostfolio/api/app/cache/cache.service';
 import { DataGatheringService } from '@ghostfolio/api/services/data-gathering.service';
 import { ExchangeRateDataService } from '@ghostfolio/api/services/exchange-rate-data.service';
 import { PrismaService } from '@ghostfolio/api/services/prisma.service';
 import { SymbolProfileService } from '@ghostfolio/api/services/symbol-profile.service';
 import {
-  DATA_GATHERING_QUEUE,
+  GATHER_ASSET_PROFILE_PROCESS,
-  GATHER_ASSET_PROFILE_PROCESS
+  GATHER_ASSET_PROFILE_PROCESS_OPTIONS
 } from '@ghostfolio/common/config';
 import { Filter } from '@ghostfolio/common/interfaces';
 import { OrderWithAccount } from '@ghostfolio/common/types';
-import { InjectQueue } from '@nestjs/bull';
 import { Injectable } from '@nestjs/common';
 import {
   AssetClass,
@@ -21,7 +19,6 @@ import {
   Type as TypeOfOrder
 } from '@prisma/client';
 import Big from 'big.js';
-import { Queue } from 'bull';
 import { endOfToday, isAfter } from 'date-fns';
 import { groupBy } from 'lodash';
 import { v4 as uuidv4 } from 'uuid';
@@ -32,11 +29,8 @@ import { Activity } from './interfaces/activities.interface';
 export class OrderService {
   public constructor(
     private readonly accountService: AccountService,
-    private readonly cacheService: CacheService,
-    @InjectQueue(DATA_GATHERING_QUEUE)
-    private readonly dataGatheringQueue: Queue,
-    private readonly exchangeRateDataService: ExchangeRateDataService,
     private readonly dataGatheringService: DataGatheringService,
+    private readonly exchangeRateDataService: ExchangeRateDataService,
     private readonly prismaService: PrismaService,
     private readonly symbolProfileService: SymbolProfileService
   ) {}
@@ -120,10 +114,14 @@ export class OrderService {
         data.SymbolProfile.connectOrCreate.create.symbol.toUpperCase();
     }
 
-    await this.dataGatheringQueue.add(GATHER_ASSET_PROFILE_PROCESS, {
+    await this.dataGatheringService.addJobToQueue(
+      GATHER_ASSET_PROFILE_PROCESS,
+      {
         dataSource: data.SymbolProfile.connectOrCreate.create.dataSource,
         symbol: data.SymbolProfile.connectOrCreate.create.symbol
-    });
+      },
+      GATHER_ASSET_PROFILE_PROCESS_OPTIONS
+    );
 
     const isDraft = isAfter(data.date as Date, endOfToday());
 
@@ -138,8 +136,6 @@ export class OrderService {
       ]);
     }
 
-    await this.cacheService.flush();
-
     delete data.accountId;
     delete data.assetClass;
     delete data.assetSubClass;
@@ -330,8 +326,6 @@ export class OrderService {
       }
     }
 
-    await this.cacheService.flush();
-
     delete data.assetClass;
     delete data.assetSubClass;
     delete data.currency;

apps/api/src/app/portfolio/portfolio-calculator.ts

@@ -56,7 +56,7 @@ export class PortfolioCalculator {
     this.currentRateService = currentRateService;
     this.orders = orders;
 
-    this.orders.sort((a, b) => a.date.localeCompare(b.date));
+    this.orders.sort((a, b) => a.date?.localeCompare(b.date));
   }
 
   public computeTransactionPoints() {
@@ -125,7 +125,7 @@ export class PortfolioCalculator {
         (transactionPointItem) => transactionPointItem.symbol !== order.symbol
       );
       newItems.push(currentTransactionPointItem);
-      newItems.sort((a, b) => a.symbol.localeCompare(b.symbol));
+      newItems.sort((a, b) => a.symbol?.localeCompare(b.symbol));
       if (lastDate !== currentDate || lastTransactionPoint === null) {
         lastTransactionPoint = {
           date: currentDate,

apps/api/src/app/user/user.service.ts

@@ -12,6 +12,7 @@ import {
 } from '@ghostfolio/common/permissions';
 import { Injectable } from '@nestjs/common';
 import { Prisma, Role, User, ViewMode } from '@prisma/client';
+import { sortBy } from 'lodash';
 
 import { UserSettingsParams } from './interfaces/user-settings-params.interface';
 import { UserSettings } from './interfaces/user-settings.interface';
@@ -185,6 +186,9 @@ export class UserService {
       }
     }
 
+    user.Account = sortBy(user.Account, (account) => {
+      return account.name;
+    });
     user.permissions = currentPermissions.sort();
 
     return user;

apps/api/src/assets/cryptocurrencies/custom.json

@@ -0,0 +1,4 @@
+{
+  "LUNA1": "Terra",
+  "UNI1": "Uniswap"
+}

apps/api/src/main.ts

@@ -20,10 +20,11 @@ async function bootstrap() {
     })
   );
 
+  const host = process.env.HOST || 'localhost';
   const port = process.env.PORT || 3333;
-  await app.listen(port, () => {
+  await app.listen(port, host, () => {
     logLogo();
-    Logger.log(`Listening at http://localhost:${port}`);
+    Logger.log(`Listening at http://${host}:${port}`);
     Logger.log('');
   });
 }

apps/api/src/services/configuration.service.ts

@@ -31,12 +31,13 @@ export class ConfigurationService {
       GOOGLE_SHEETS_ACCOUNT: str({ default: '' }),
       GOOGLE_SHEETS_ID: str({ default: '' }),
       GOOGLE_SHEETS_PRIVATE_KEY: str({ default: '' }),
+      HOST: host({ default: 'localhost' }),
       JWT_SECRET_KEY: str({}),
       MAX_ITEM_IN_CACHE: num({ default: 9999 }),
       MAX_ORDERS_TO_IMPORT: num({ default: Number.MAX_SAFE_INTEGER }),
       PORT: port({ default: 3333 }),
       RAKUTEN_RAPID_API_KEY: str({ default: '' }),
-      REDIS_HOST: str({ default: 'localhost' }),
+      REDIS_HOST: host({ default: 'localhost' }),
       REDIS_PASSWORD: str({ default: '' }),
       REDIS_PORT: port({ default: 6379 }),
       ROOT_URL: str({ default: 'http://localhost:4200' }),

apps/api/src/services/cron.service.ts

@@ -1,11 +1,9 @@
 import {
-  DATA_GATHERING_QUEUE,
+  GATHER_ASSET_PROFILE_PROCESS,
-  GATHER_ASSET_PROFILE_PROCESS
+  GATHER_ASSET_PROFILE_PROCESS_OPTIONS
 } from '@ghostfolio/common/config';
-import { InjectQueue } from '@nestjs/bull';
 import { Injectable } from '@nestjs/common';
 import { Cron, CronExpression } from '@nestjs/schedule';
-import { Queue } from 'bull';
 
 import { DataGatheringService } from './data-gathering.service';
 import { ExchangeRateDataService } from './exchange-rate-data.service';
@@ -14,15 +12,13 @@ import { TwitterBotService } from './twitter-bot/twitter-bot.service';
 @Injectable()
 export class CronService {
   public constructor(
-    @InjectQueue(DATA_GATHERING_QUEUE)
-    private readonly dataGatheringQueue: Queue,
     private readonly dataGatheringService: DataGatheringService,
     private readonly exchangeRateDataService: ExchangeRateDataService,
     private readonly twitterBotService: TwitterBotService
   ) {}
 
-  @Cron(CronExpression.EVERY_MINUTE)
+  @Cron(CronExpression.EVERY_HOUR)
-  public async runEveryMinute() {
+  public async runEveryHour() {
     await this.dataGatheringService.gather7Days();
   }
 
@@ -41,10 +37,14 @@ export class CronService {
     const uniqueAssets = await this.dataGatheringService.getUniqueAssets();
 
     for (const { dataSource, symbol } of uniqueAssets) {
-      await this.dataGatheringQueue.add(GATHER_ASSET_PROFILE_PROCESS, {
+      await this.dataGatheringService.addJobToQueue(
+        GATHER_ASSET_PROFILE_PROCESS,
+        {
           dataSource,
           symbol
-      });
+        },
+        GATHER_ASSET_PROFILE_PROCESS_OPTIONS
+      );
     }
   }
 }

apps/api/src/services/cryptocurrency/cryptocurrency.service.ts

@@ -1,8 +1,7 @@
 import { Injectable } from '@nestjs/common';
 
-const cryptocurrencies = require('cryptocurrencies');
+const cryptocurrencies = require('../../assets/cryptocurrencies/cryptocurrencies.json');
-
+const customCryptocurrencies = require('../../assets/cryptocurrencies/custom.json');
-const customCryptocurrencies = require('./custom-cryptocurrencies.json');
 
 @Injectable()
 export class CryptocurrencyService {
@@ -18,7 +17,7 @@ export class CryptocurrencyService {
   private getCryptocurrencies() {
     if (!this.combinedCryptocurrencies) {
       this.combinedCryptocurrencies = [
-        ...cryptocurrencies.symbols(),
+        ...Object.keys(cryptocurrencies),
         ...Object.keys(customCryptocurrencies)
       ];
     }

apps/api/src/services/cryptocurrency/custom-cryptocurrencies.json

@@ -1,14 +0,0 @@
-{
-  "1INCH": "1inch",
-  "ALGO": "Algorand",
-  "ATOM": "Cosmos",
-  "AVAX": "Avalanche",
-  "DOT": "Polkadot",
-  "LUNA1": "Terra",
-  "MATIC": "Polygon",
-  "MINA": "Mina Protocol",
-  "RUNE": "THORChain",
-  "SHIB": "Shiba Inu",
-  "SOL": "Solana",
-  "UNI3": "Uniswap"
-}

apps/api/src/services/data-gathering.module.ts

@@ -6,6 +6,7 @@ import { PrismaModule } from '@ghostfolio/api/services/prisma.module';
 import { DATA_GATHERING_QUEUE } from '@ghostfolio/common/config';
 import { BullModule } from '@nestjs/bull';
 import { Module } from '@nestjs/common';
+import ms from 'ms';
 
 import { DataGatheringProcessor } from './data-gathering.processor';
 import { ExchangeRateDataModule } from './exchange-rate-data.module';
@@ -14,6 +15,10 @@ import { SymbolProfileModule } from './symbol-profile.module';
 @Module({
   imports: [
     BullModule.registerQueue({
+      limiter: {
+        duration: ms('5 seconds'),
+        max: 1
+      },
       name: DATA_GATHERING_QUEUE
     }),
     ConfigurationModule,

apps/api/src/services/data-gathering.processor.ts

@@ -1,19 +1,34 @@
 import {
   DATA_GATHERING_QUEUE,
-  GATHER_ASSET_PROFILE_PROCESS
+  GATHER_ASSET_PROFILE_PROCESS,
+  GATHER_HISTORICAL_MARKET_DATA_PROCESS
 } from '@ghostfolio/common/config';
+import { DATE_FORMAT } from '@ghostfolio/common/helper';
 import { UniqueAsset } from '@ghostfolio/common/interfaces';
 import { Process, Processor } from '@nestjs/bull';
 import { Injectable, Logger } from '@nestjs/common';
 import { Job } from 'bull';
+import {
+  format,
+  getDate,
+  getMonth,
+  getYear,
+  isBefore,
+  parseISO
+} from 'date-fns';
 
 import { DataGatheringService } from './data-gathering.service';
+import { DataProviderService } from './data-provider/data-provider.service';
+import { IDataGatheringItem } from './interfaces/interfaces';
+import { PrismaService } from './prisma.service';
 
 @Injectable()
 @Processor(DATA_GATHERING_QUEUE)
 export class DataGatheringProcessor {
   public constructor(
-    private readonly dataGatheringService: DataGatheringService
+    private readonly dataGatheringService: DataGatheringService,
+    private readonly dataProviderService: DataProviderService,
+    private readonly prismaService: PrismaService
   ) {}
 
   @Process(GATHER_ASSET_PROFILE_PROCESS)
@@ -21,7 +36,93 @@ export class DataGatheringProcessor {
     try {
       await this.dataGatheringService.gatherAssetProfiles([job.data]);
     } catch (error) {
-      Logger.error(error, 'DataGatheringProcessor');
+      Logger.error(
+        error,
+        `DataGatheringProcessor (${GATHER_ASSET_PROFILE_PROCESS})`
+      );
+
+      throw new Error(error);
+    }
+  }
+
+  @Process(GATHER_HISTORICAL_MARKET_DATA_PROCESS)
+  public async gatherHistoricalMarketData(job: Job<IDataGatheringItem>) {
+    try {
+      const { dataSource, date, symbol } = job.data;
+
+      const historicalData = await this.dataProviderService.getHistoricalRaw(
+        [{ dataSource, symbol }],
+        parseISO(<string>(<unknown>date)),
+        new Date()
+      );
+
+      let currentDate = parseISO(<string>(<unknown>date));
+      let lastMarketPrice: number;
+
+      while (
+        isBefore(
+          currentDate,
+          new Date(
+            Date.UTC(
+              getYear(new Date()),
+              getMonth(new Date()),
+              getDate(new Date()),
+              0
+            )
+          )
+        )
+      ) {
+        if (
+          historicalData[symbol]?.[format(currentDate, DATE_FORMAT)]
+            ?.marketPrice
+        ) {
+          lastMarketPrice =
+            historicalData[symbol]?.[format(currentDate, DATE_FORMAT)]
+              ?.marketPrice;
+        }
+
+        if (lastMarketPrice) {
+          try {
+            await this.prismaService.marketData.create({
+              data: {
+                dataSource,
+                symbol,
+                date: new Date(
+                  Date.UTC(
+                    getYear(currentDate),
+                    getMonth(currentDate),
+                    getDate(currentDate),
+                    0
+                  )
+                ),
+                marketPrice: lastMarketPrice
+              }
+            });
+          } catch {}
+        }
+
+        // Count month one up for iteration
+        currentDate = new Date(
+          Date.UTC(
+            getYear(currentDate),
+            getMonth(currentDate),
+            getDate(currentDate) + 1,
+            0
+          )
+        );
+      }
+
+      Logger.log(
+        `Historical market data gathering has been completed for ${symbol} (${dataSource}).`,
+        `DataGatheringProcessor (${GATHER_HISTORICAL_MARKET_DATA_PROCESS})`
+      );
+    } catch (error) {
+      Logger.error(
+        error,
+        `DataGatheringProcessor (${GATHER_HISTORICAL_MARKET_DATA_PROCESS})`
+      );
+
+      throw new Error(error);
     }
   }
 }

apps/api/src/services/data-gathering.service.ts

@@ -1,21 +1,17 @@
 import { SymbolProfileService } from '@ghostfolio/api/services/symbol-profile.service';
 import {
-  PROPERTY_LAST_DATA_GATHERING,
+  DATA_GATHERING_QUEUE,
-  PROPERTY_LOCKED_DATA_GATHERING
+  GATHER_HISTORICAL_MARKET_DATA_PROCESS,
+  GATHER_HISTORICAL_MARKET_DATA_PROCESS_OPTIONS,
+  QUEUE_JOB_STATUS_LIST
 } from '@ghostfolio/common/config';
 import { DATE_FORMAT, resetHours } from '@ghostfolio/common/helper';
 import { UniqueAsset } from '@ghostfolio/common/interfaces';
+import { InjectQueue } from '@nestjs/bull';
 import { Inject, Injectable, Logger } from '@nestjs/common';
 import { DataSource } from '@prisma/client';
-import {
+import { JobOptions, Queue } from 'bull';
-  differenceInHours,
+import { format, subDays } from 'date-fns';
-  format,
-  getDate,
-  getMonth,
-  getYear,
-  isBefore,
-  subDays
-} from 'date-fns';
 
 import { DataProviderService } from './data-provider/data-provider.service';
 import { DataEnhancerInterface } from './data-provider/interfaces/data-enhancer.interface';
@@ -25,167 +21,48 @@ import { PrismaService } from './prisma.service';
 
 @Injectable()
 export class DataGatheringService {
-  private dataGatheringProgress: number;
-
   public constructor(
     @Inject('DataEnhancers')
     private readonly dataEnhancers: DataEnhancerInterface[],
+    @InjectQueue(DATA_GATHERING_QUEUE)
+    private readonly dataGatheringQueue: Queue,
     private readonly dataProviderService: DataProviderService,
     private readonly exchangeRateDataService: ExchangeRateDataService,
     private readonly prismaService: PrismaService,
     private readonly symbolProfileService: SymbolProfileService
   ) {}
 
-  public async gather7Days() {
+  public async addJobToQueue(name: string, data: any, options?: JobOptions) {
-    const isDataGatheringNeeded = await this.isDataGatheringNeeded();
+    const hasJob = await this.hasJob(name, data);
-
-    if (isDataGatheringNeeded) {
-      Logger.log('7d data gathering has been started.', 'DataGatheringService');
-      console.time('data-gathering-7d');
-
-      await this.prismaService.property.create({
-        data: {
-          key: PROPERTY_LOCKED_DATA_GATHERING,
-          value: new Date().toISOString()
-        }
-      });
-
-      const symbols = await this.getSymbols7D();
-
-      try {
-        await this.gatherSymbols(symbols);
-
-        await this.prismaService.property.upsert({
-          create: {
-            key: PROPERTY_LAST_DATA_GATHERING,
-            value: new Date().toISOString()
-          },
-          update: { value: new Date().toISOString() },
-          where: { key: PROPERTY_LAST_DATA_GATHERING }
-        });
-      } catch (error) {
-        Logger.error(error, 'DataGatheringService');
-      }
-
-      await this.prismaService.property.delete({
-        where: {
-          key: PROPERTY_LOCKED_DATA_GATHERING
-        }
-      });
 
+    if (hasJob) {
       Logger.log(
-        '7d data gathering has been completed.',
+        `Job ${name} with data ${JSON.stringify(data)} already exists.`,
         'DataGatheringService'
       );
-      console.timeEnd('data-gathering-7d');
+    } else {
+      return this.dataGatheringQueue.add(name, data, options);
     }
   }
 
+  public async gather7Days() {
+    const dataGatheringItems = await this.getSymbols7D();
+    await this.gatherSymbols(dataGatheringItems);
+  }
+
   public async gatherMax() {
-    const isDataGatheringLocked = await this.prismaService.property.findUnique({
+    const dataGatheringItems = await this.getSymbolsMax();
-      where: { key: PROPERTY_LOCKED_DATA_GATHERING }
+    await this.gatherSymbols(dataGatheringItems);
-    });
-
-    if (!isDataGatheringLocked) {
-      Logger.log(
-        'Max data gathering has been started.',
-        'DataGatheringService'
-      );
-      console.time('data-gathering-max');
-
-      await this.prismaService.property.create({
-        data: {
-          key: PROPERTY_LOCKED_DATA_GATHERING,
-          value: new Date().toISOString()
-        }
-      });
-
-      const symbols = await this.getSymbolsMax();
-
-      try {
-        await this.gatherSymbols(symbols);
-
-        await this.prismaService.property.upsert({
-          create: {
-            key: PROPERTY_LAST_DATA_GATHERING,
-            value: new Date().toISOString()
-          },
-          update: { value: new Date().toISOString() },
-          where: { key: PROPERTY_LAST_DATA_GATHERING }
-        });
-      } catch (error) {
-        Logger.error(error, 'DataGatheringService');
-      }
-
-      await this.prismaService.property.delete({
-        where: {
-          key: PROPERTY_LOCKED_DATA_GATHERING
-        }
-      });
-
-      Logger.log(
-        'Max data gathering has been completed.',
-        'DataGatheringService'
-      );
-      console.timeEnd('data-gathering-max');
-    }
   }
 
   public async gatherSymbol({ dataSource, symbol }: UniqueAsset) {
-    const isDataGatheringLocked = await this.prismaService.property.findUnique({
+    const symbols = (await this.getSymbolsMax()).filter((dataGatheringItem) => {
-      where: { key: PROPERTY_LOCKED_DATA_GATHERING }
-    });
-
-    if (!isDataGatheringLocked) {
-      Logger.log(
-        `Symbol data gathering for ${symbol} has been started.`,
-        'DataGatheringService'
-      );
-      console.time('data-gathering-symbol');
-
-      await this.prismaService.property.create({
-        data: {
-          key: PROPERTY_LOCKED_DATA_GATHERING,
-          value: new Date().toISOString()
-        }
-      });
-
-      const symbols = (await this.getSymbolsMax()).filter(
-        (dataGatheringItem) => {
       return (
         dataGatheringItem.dataSource === dataSource &&
         dataGatheringItem.symbol === symbol
       );
-        }
+    });
-      );
-
-      try {
     await this.gatherSymbols(symbols);
-
-        await this.prismaService.property.upsert({
-          create: {
-            key: PROPERTY_LAST_DATA_GATHERING,
-            value: new Date().toISOString()
-          },
-          update: { value: new Date().toISOString() },
-          where: { key: PROPERTY_LAST_DATA_GATHERING }
-        });
-      } catch (error) {
-        Logger.error(error, 'DataGatheringService');
-      }
-
-      await this.prismaService.property.delete({
-        where: {
-          key: PROPERTY_LOCKED_DATA_GATHERING
-        }
-      });
-
-      Logger.log(
-        `Symbol data gathering for ${symbol} has been completed.`,
-        'DataGatheringService'
-      );
-      console.timeEnd('data-gathering-symbol');
-    }
   }
 
   public async gatherSymbolForDate({
@@ -235,15 +112,6 @@ export class DataGatheringService {
       uniqueAssets = await this.getUniqueAssets();
     }
 
-    Logger.log(
-      `Asset profile data gathering has been started for ${uniqueAssets
-        .map(({ dataSource, symbol }) => {
-          return `${symbol} (${dataSource})`;
-        })
-        .join(',')}.`,
-      'DataGatheringService'
-    );
-
     const assetProfiles = await this.dataProviderService.getAssetProfiles(
       uniqueAssets
     );
@@ -334,136 +202,21 @@ export class DataGatheringService {
   }
 
   public async gatherSymbols(aSymbolsWithStartDate: IDataGatheringItem[]) {
-    let hasError = false;
-    let symbolCounter = 0;
-
     for (const { dataSource, date, symbol } of aSymbolsWithStartDate) {
       if (dataSource === 'MANUAL') {
         continue;
       }
 
-      this.dataGatheringProgress = symbolCounter / aSymbolsWithStartDate.length;
+      await this.addJobToQueue(
-
+        GATHER_HISTORICAL_MARKET_DATA_PROCESS,
-      try {
+        {
-        const historicalData = await this.dataProviderService.getHistoricalRaw(
-          [{ dataSource, symbol }],
-          date,
-          new Date()
-        );
-
-        let currentDate = date;
-        let lastMarketPrice: number;
-
-        while (
-          isBefore(
-            currentDate,
-            new Date(
-              Date.UTC(
-                getYear(new Date()),
-                getMonth(new Date()),
-                getDate(new Date()),
-                0
-              )
-            )
-          )
-        ) {
-          if (
-            historicalData[symbol]?.[format(currentDate, DATE_FORMAT)]
-              ?.marketPrice
-          ) {
-            lastMarketPrice =
-              historicalData[symbol]?.[format(currentDate, DATE_FORMAT)]
-                ?.marketPrice;
-          }
-
-          if (lastMarketPrice) {
-            try {
-              await this.prismaService.marketData.create({
-                data: {
           dataSource,
-                  symbol,
+          date,
-                  date: new Date(
+          symbol
-                    Date.UTC(
+        },
-                      getYear(currentDate),
+        GATHER_HISTORICAL_MARKET_DATA_PROCESS_OPTIONS
-                      getMonth(currentDate),
-                      getDate(currentDate),
-                      0
-                    )
-                  ),
-                  marketPrice: lastMarketPrice
-                }
-              });
-            } catch {}
-          } else {
-            Logger.warn(
-              `Failed to gather data for symbol ${symbol} from ${dataSource} at ${format(
-                currentDate,
-                DATE_FORMAT
-              )}.`,
-              'DataGatheringService'
       );
     }
-
-          // Count month one up for iteration
-          currentDate = new Date(
-            Date.UTC(
-              getYear(currentDate),
-              getMonth(currentDate),
-              getDate(currentDate) + 1,
-              0
-            )
-          );
-        }
-      } catch (error) {
-        hasError = true;
-        Logger.error(error, 'DataGatheringService');
-      }
-
-      if (symbolCounter > 0 && symbolCounter % 100 === 0) {
-        Logger.log(
-          `Data gathering progress: ${(
-            this.dataGatheringProgress * 100
-          ).toFixed(2)}%`,
-          'DataGatheringService'
-        );
-      }
-
-      symbolCounter += 1;
-    }
-
-    await this.exchangeRateDataService.initialize();
-
-    if (hasError) {
-      throw '';
-    }
-  }
-
-  public async getDataGatheringProgress() {
-    const isInProgress = await this.getIsInProgress();
-
-    if (isInProgress) {
-      return this.dataGatheringProgress;
-    }
-
-    return undefined;
-  }
-
-  public async getIsInProgress() {
-    return await this.prismaService.property.findUnique({
-      where: { key: PROPERTY_LOCKED_DATA_GATHERING }
-    });
-  }
-
-  public async getLastDataGathering() {
-    const lastDataGathering = await this.prismaService.property.findUnique({
-      where: { key: PROPERTY_LAST_DATA_GATHERING }
-    });
-
-    if (lastDataGathering?.value) {
-      return new Date(lastDataGathering.value);
-    }
-
-    return undefined;
   }
 
   public async getSymbolsMax(): Promise<IDataGatheringItem[]> {
@@ -534,19 +287,6 @@ export class DataGatheringService {
       });
   }
 
-  public async reset() {
-    Logger.log('Data gathering has been reset.', 'DataGatheringService');
-
-    await this.prismaService.property.deleteMany({
-      where: {
-        OR: [
-          { key: PROPERTY_LAST_DATA_GATHERING },
-          { key: PROPERTY_LOCKED_DATA_GATHERING }
-        ]
-      }
-    });
-  }
-
   private async getSymbols7D(): Promise<IDataGatheringItem[]> {
     const startDate = subDays(resetHours(new Date()), 7);
 
@@ -610,15 +350,17 @@ export class DataGatheringService {
     return [...currencyPairsToGather, ...symbolProfilesToGather];
   }
 
-  private async isDataGatheringNeeded() {
+  private async hasJob(name: string, data: any) {
-    const lastDataGathering = await this.getLastDataGathering();
+    const jobs = await this.dataGatheringQueue.getJobs(
+      QUEUE_JOB_STATUS_LIST.filter((status) => {
+        return status !== 'completed';
+      })
+    );
 
-    const isDataGatheringLocked = await this.prismaService.property.findUnique({
+    return jobs.some((job) => {
-      where: { key: PROPERTY_LOCKED_DATA_GATHERING }
+      return (
+        job.name === name && JSON.stringify(job.data) === JSON.stringify(data)
+      );
     });
-
-    const diffInHours = differenceInHours(new Date(), lastDataGathering);
-
-    return (diffInHours >= 1 || !lastDataGathering) && !isDataGatheringLocked;
   }
 }

apps/api/src/services/data-provider/alpha-vantage/alpha-vantage.service.ts

@@ -9,7 +9,7 @@ import { DATE_FORMAT } from '@ghostfolio/common/helper';
 import { Granularity } from '@ghostfolio/common/types';
 import { Injectable, Logger } from '@nestjs/common';
 import { DataSource, SymbolProfile } from '@prisma/client';
-import { isAfter, isBefore, parse } from 'date-fns';
+import { format, isAfter, isBefore, parse } from 'date-fns';
 
 import { IAlphaVantageHistoricalResponse } from './interfaces/interfaces';
 
@@ -76,9 +76,12 @@ export class AlphaVantageService implements DataProviderInterface {
 
       return response;
     } catch (error) {
-      Logger.error(error, 'AlphaVantageService');
+      throw new Error(
-
+        `Could not get historical market data for ${aSymbol} (${this.getName()}) from ${format(
-      return {};
+          from,
+          DATE_FORMAT
+        )} to ${format(to, DATE_FORMAT)}: [${error.name}] ${error.message}`
+      );
     }
   }
 

apps/api/src/services/data-provider/eod-historical-data/eod-historical-data.service.ts

@@ -72,10 +72,13 @@ export class EodHistoricalDataService implements DataProviderInterface {
         { [aSymbol]: {} }
       );
     } catch (error) {
-      Logger.error(error, 'EodHistoricalDataService');
+      throw new Error(
+        `Could not get historical market data for ${aSymbol} (${this.getName()}) from ${format(
+          from,
+          DATE_FORMAT
+        )} to ${format(to, DATE_FORMAT)}: [${error.name}] ${error.message}`
+      );
     }
-
-    return {};
   }
 
   public getName(): DataSource {

apps/api/src/services/data-provider/ghostfolio-scraper-api/ghostfolio-scraper-api.service.ts

@@ -87,10 +87,13 @@ export class GhostfolioScraperApiService implements DataProviderInterface {
         }
       };
     } catch (error) {
-      Logger.error(error, 'GhostfolioScraperApiService');
+      throw new Error(
+        `Could not get historical market data for ${aSymbol} (${this.getName()}) from ${format(
+          from,
+          DATE_FORMAT
+        )} to ${format(to, DATE_FORMAT)}: [${error.name}] ${error.message}`
+      );
     }
-
-    return {};
   }
 
   public getName(): DataSource {

apps/api/src/services/data-provider/google-sheets/google-sheets.service.ts

@@ -71,10 +71,13 @@ export class GoogleSheetsService implements DataProviderInterface {
         [symbol]: historicalData
       };
     } catch (error) {
-      Logger.error(error, 'GoogleSheetsService');
+      throw new Error(
+        `Could not get historical market data for ${aSymbol} (${this.getName()}) from ${format(
+          from,
+          DATE_FORMAT
+        )} to ${format(to, DATE_FORMAT)}: [${error.name}] ${error.message}`
+      );
     }
-
-    return {};
   }
 
   public getName(): DataSource {

apps/api/src/services/data-provider/rakuten-rapid-api/rakuten-rapid-api.service.ts

@@ -90,7 +90,14 @@ export class RakutenRapidApiService implements DataProviderInterface {
           }
         };
       }
-    } catch (error) {}
+    } catch (error) {
+      throw new Error(
+        `Could not get historical market data for ${aSymbol} (${this.getName()}) from ${format(
+          from,
+          DATE_FORMAT
+        )} to ${format(to, DATE_FORMAT)}: [${error.name}] ${error.message}`
+      );
+    }
 
     return {};
   }

apps/api/src/services/data-provider/yahoo-finance/yahoo-finance.service.ts

@@ -131,7 +131,13 @@ export class YahooFinanceService implements DataProviderInterface {
       if (url) {
         response.url = url;
       }
-    } catch {}
+    } catch (error) {
+      throw new Error(
+        `Could not get asset profile for ${aSymbol} (${this.getName()}): [${
+          error.name
+        }] ${error.message}`
+      );
+    }
 
     return response;
   }
@@ -185,12 +191,12 @@ export class YahooFinanceService implements DataProviderInterface {
 
       return response;
     } catch (error) {
-      Logger.warn(
+      throw new Error(
-        `Skipping yahooFinance2.getHistorical("${aSymbol}"): [${error.name}] ${error.message}`,
+        `Could not get historical market data for ${aSymbol} (${this.getName()}) from ${format(
-        'YahooFinanceService'
+          from,
+          DATE_FORMAT
+        )} to ${format(to, DATE_FORMAT)}: [${error.name}] ${error.message}`
       );
-
-      return {};
     }
   }
 

apps/api/src/services/interfaces/interfaces.ts

@@ -1,3 +1,4 @@
+import { UniqueAsset } from '@ghostfolio/common/interfaces';
 import { MarketState } from '@ghostfolio/common/types';
 import {
   Account,
@@ -32,8 +33,6 @@ export interface IDataProviderResponse {
   marketState: MarketState;
 }
 
-export interface IDataGatheringItem {
+export interface IDataGatheringItem extends UniqueAsset {
-  dataSource: DataSource;
   date?: Date;
-  symbol: string;
 }

apps/client/src/app/components/admin-jobs/admin-jobs.component.ts

@@ -0,0 +1,116 @@
+import {
+  ChangeDetectionStrategy,
+  ChangeDetectorRef,
+  Component,
+  OnDestroy,
+  OnInit
+} from '@angular/core';
+import { FormBuilder, FormGroup } from '@angular/forms';
+import { AdminService } from '@ghostfolio/client/services/admin.service';
+import { UserService } from '@ghostfolio/client/services/user/user.service';
+import { QUEUE_JOB_STATUS_LIST } from '@ghostfolio/common/config';
+import { getDateWithTimeFormatString } from '@ghostfolio/common/helper';
+import { AdminJobs, User } from '@ghostfolio/common/interfaces';
+import { JobStatus } from 'bull';
+import { Subject } from 'rxjs';
+import { takeUntil } from 'rxjs/operators';
+
+@Component({
+  changeDetection: ChangeDetectionStrategy.OnPush,
+  selector: 'gf-admin-jobs',
+  styleUrls: ['./admin-jobs.scss'],
+  templateUrl: './admin-jobs.html'
+})
+export class AdminJobsComponent implements OnDestroy, OnInit {
+  public defaultDateTimeFormat: string;
+  public filterForm: FormGroup;
+  public jobs: AdminJobs['jobs'] = [];
+  public statusFilterOptions = QUEUE_JOB_STATUS_LIST;
+  public user: User;
+
+  private unsubscribeSubject = new Subject<void>();
+
+  /**
+   * @constructor
+   */
+  public constructor(
+    private adminService: AdminService,
+    private changeDetectorRef: ChangeDetectorRef,
+    private formBuilder: FormBuilder,
+    private userService: UserService
+  ) {
+    this.userService.stateChanged
+      .pipe(takeUntil(this.unsubscribeSubject))
+      .subscribe((state) => {
+        if (state?.user) {
+          this.user = state.user;
+
+          this.defaultDateTimeFormat = getDateWithTimeFormatString(
+            this.user.settings.locale
+          );
+        }
+      });
+  }
+
+  /**
+   * Initializes the controller
+   */
+  public ngOnInit() {
+    this.filterForm = this.formBuilder.group({
+      status: []
+    });
+
+    this.filterForm.valueChanges
+      .pipe(takeUntil(this.unsubscribeSubject))
+      .subscribe(() => {
+        const currentFilter = this.filterForm.get('status').value;
+        this.fetchJobs(currentFilter ? [currentFilter] : undefined);
+      });
+
+    this.fetchJobs();
+  }
+
+  public onDeleteJob(aId: string) {
+    this.adminService
+      .deleteJob(aId)
+      .pipe(takeUntil(this.unsubscribeSubject))
+      .subscribe(() => {
+        this.fetchJobs();
+      });
+  }
+
+  public onDeleteJobs() {
+    const currentFilter = this.filterForm.get('status').value;
+
+    this.adminService
+      .deleteJobs({ status: currentFilter ? [currentFilter] : undefined })
+      .pipe(takeUntil(this.unsubscribeSubject))
+      .subscribe(() => {
+        this.fetchJobs(currentFilter ? [currentFilter] : undefined);
+      });
+  }
+
+  public onViewData(aData: AdminJobs['jobs'][0]['data']) {
+    alert(JSON.stringify(aData, null, '  '));
+  }
+
+  public onViewStacktrace(aStacktrace: AdminJobs['jobs'][0]['stacktrace']) {
+    alert(JSON.stringify(aStacktrace, null, '  '));
+  }
+
+  public ngOnDestroy() {
+    this.unsubscribeSubject.next();
+    this.unsubscribeSubject.complete();
+  }
+
+  private fetchJobs(aStatus?: JobStatus[]) {
+    this.adminService
+      .fetchJobs({ status: aStatus })
+      .pipe(takeUntil(this.unsubscribeSubject))
+      .subscribe(({ jobs }) => {
+        this.jobs = jobs;
+
+        this.changeDetectorRef.markForCheck();
+      });
+  }
+}

apps/client/src/app/components/admin-jobs/admin-jobs.html

@@ -0,0 +1,130 @@
+<div class="container">
+  <div class="row">
+    <div class="col">
+      <form class="align-items-center d-flex" [formGroup]="filterForm">
+        <mat-form-field appearance="outline" class="flex-grow-1">
+          <mat-select formControlName="status">
+            <mat-option></mat-option>
+            <mat-option
+              *ngFor="let statusFilterOption of statusFilterOptions"
+              [value]="statusFilterOption"
+              >{{ statusFilterOption }}</mat-option
+            >
+          </mat-select>
+        </mat-form-field>
+        <button
+          class="ml-1"
+          color="warn"
+          mat-flat-button
+          (click)="onDeleteJobs()"
+        >
+          <span i18n>Delete Jobs</span>
+        </button>
+      </form>
+      <table class="gf-table w-100">
+        <thead>
+          <tr class="mat-header-row">
+            <th class="mat-header-cell px-1 py-2 text-right" i18n>#</th>
+            <th class="mat-header-cell px-1 py-2" i18n>Type</th>
+            <th class="mat-header-cell px-1 py-2" i18n>Symbol</th>
+            <th class="mat-header-cell px-1 py-2" i18n>Data Source</th>
+            <th class="mat-header-cell px-1 py-2 text-right" i18n>Attempts</th>
+            <th class="mat-header-cell px-1 py-2" i18n>Created</th>
+            <th class="mat-header-cell px-1 py-2" i18n>Finished</th>
+            <th class="mat-header-cell px-1 py-2" i18n>Status</th>
+            <th class="mat-header-cell px-1 py-2"></th>
+          </tr>
+        </thead>
+        <tbody>
+          <ng-container *ngFor="let job of jobs">
+            <tr class="mat-row">
+              <td class="mat-cell px-1 py-2 text-right">{{ job.id }}</td>
+              <td class="mat-cell px-1 py-2">
+                <span class="align-items-center d-flex">
+                  <ion-icon
+                    class="mr-1"
+                    name="arrow-down-circle-outline"
+                  ></ion-icon>
+                  <ng-container *ngIf="job.name === 'GATHER_ASSET_PROFILE'">
+                    <span i18n>Asset Profile</span>
+                  </ng-container>
+                  <ng-container
+                    *ngIf="job.name === 'GATHER_HISTORICAL_MARKET_DATA'"
+                  >
+                    <span i18n>Historical Market Data</span>
+                  </ng-container>
+                </span>
+              </td>
+              <td class="mat-cell px-1 py-2">{{ job.data?.symbol }}</td>
+              <td class="mat-cell px-1 py-2">{{ job.data?.dataSource }}</td>
+              <td class="mat-cell px-1 py-2 text-right">
+                {{ job.attemptsMade }}
+              </td>
+              <td class="mat-cell px-1 py-2">
+                {{ job.timestamp | date: defaultDateTimeFormat }}
+              </td>
+              <td class="mat-cell px-1 py-2">
+                {{ job.finishedOn | date: defaultDateTimeFormat }}
+              </td>
+              <td class="mat-cell px-1 py-2">
+                <ion-icon
+                  *ngIf="job.state === 'active'"
+                  name="play-outline"
+                ></ion-icon>
+                <ion-icon
+                  *ngIf="job.state === 'completed'"
+                  class="text-success"
+                  name="checkmark-circle-outline"
+                ></ion-icon>
+                <ion-icon
+                  *ngIf="job.state === 'delayed'"
+                  name="time-outline"
+                  [ngClass]="{ 'text-danger': job.stacktrace?.length > 0 }"
+                ></ion-icon>
+                <ion-icon
+                  *ngIf="job.state === 'failed'"
+                  class="text-danger"
+                  name="alert-circle-outline"
+                ></ion-icon>
+                <ion-icon
+                  *ngIf="job.state === 'paused'"
+                  name="pause-outline"
+                ></ion-icon>
+                <ion-icon
+                  *ngIf="job.state === 'waiting'"
+                  name="cafe-outline"
+                ></ion-icon>
+              </td>
+              <td class="mat-cell px-1 py-2">
+                <button
+                  class="mx-1 no-min-width px-2"
+                  mat-button
+                  [matMenuTriggerFor]="accountMenu"
+                  (click)="$event.stopPropagation()"
+                >
+                  <ion-icon name="ellipsis-vertical"></ion-icon>
+                </button>
+                <mat-menu #accountMenu="matMenu" xPosition="before">
+                  <button i18n mat-menu-item (click)="onViewData(job.data)">
+                    View Data
+                  </button>
+                  <button
+                    i18n
+                    mat-menu-item
+                    [disabled]="job.stacktrace?.length <= 0"
+                    (click)="onViewStacktrace(job.stacktrace)"
+                  >
+                    View Stacktrace
+                  </button>
+                  <button i18n mat-menu-item (click)="onDeleteJob(job.id)">
+                    Delete Job
+                  </button>
+                </mat-menu>
+              </td>
+            </tr>
+          </ng-container>
+        </tbody>
+      </table>
+    </div>
+  </div>
+</div>

apps/client/src/app/components/admin-jobs/admin-jobs.module.ts

@@ -0,0 +1,22 @@
+import { CommonModule } from '@angular/common';
+import { CUSTOM_ELEMENTS_SCHEMA, NgModule } from '@angular/core';
+import { FormsModule, ReactiveFormsModule } from '@angular/forms';
+import { MatButtonModule } from '@angular/material/button';
+import { MatMenuModule } from '@angular/material/menu';
+import { MatSelectModule } from '@angular/material/select';
+
+import { AdminJobsComponent } from './admin-jobs.component';
+
+@NgModule({
+  declarations: [AdminJobsComponent],
+  imports: [
+    CommonModule,
+    FormsModule,
+    MatButtonModule,
+    MatMenuModule,
+    MatSelectModule,
+    ReactiveFormsModule
+  ],
+  schemas: [CUSTOM_ELEMENTS_SCHEMA]
+})
+export class GfAdminJobsModule {}

apps/client/src/app/components/admin-jobs/admin-jobs.scss

@@ -0,0 +1,5 @@
+@import '~apps/client/src/styles/ghostfolio-style';
+
+:host {
+  display: block;
+}

apps/client/src/app/components/admin-overview/admin-overview.component.ts

@@ -15,7 +15,6 @@ import { hasPermission, permissions } from '@ghostfolio/common/permissions';
 import {
   differenceInSeconds,
   formatDistanceToNowStrict,
-  isValid,
   parseISO
 } from 'date-fns';
 import { uniq } from 'lodash';
@@ -32,14 +31,11 @@ export class AdminOverviewComponent implements OnDestroy, OnInit {
   public couponDuration: StringValue = '30 days';
   public coupons: Coupon[];
   public customCurrencies: string[];
-  public dataGatheringInProgress: boolean;
-  public dataGatheringProgress: number;
   public exchangeRates: { label1: string; label2: string; value: number }[];
   public hasPermissionForSubscription: boolean;
   public hasPermissionForSystemMessage: boolean;
   public hasPermissionToToggleReadOnlyMode: boolean;
   public info: InfoItem;
-  public lastDataGathering: string;
   public transactionCount: number;
   public userCount: number;
   public user: User;
@@ -128,7 +124,7 @@ export class AdminOverviewComponent implements OnDestroy, OnInit {
   public onDeleteCoupon(aCouponCode: string) {
     const confirmation = confirm('Do you really want to delete this coupon?');
 
-    if (confirmation) {
+    if (confirmation === true) {
       const coupons = this.coupons.filter((coupon) => {
         return coupon.code !== aCouponCode;
       });
@@ -139,7 +135,7 @@ export class AdminOverviewComponent implements OnDestroy, OnInit {
   public onDeleteCurrency(aCurrency: string) {
     const confirmation = confirm('Do you really want to delete this currency?');
 
-    if (confirmation) {
+    if (confirmation === true) {
       const currencies = this.customCurrencies.filter((currency) => {
         return currency !== aCurrency;
       });
@@ -152,6 +148,9 @@ export class AdminOverviewComponent implements OnDestroy, OnInit {
   }
 
   public onFlushCache() {
+    const confirmation = confirm('Do you really want to flush the cache?');
+
+    if (confirmation === true) {
       this.cacheService
         .flush()
         .pipe(takeUntil(this.unsubscribeSubject))
@@ -161,13 +160,20 @@ export class AdminOverviewComponent implements OnDestroy, OnInit {
           }, 300);
         });
     }
+  }
+
+  public onGather7Days() {
+    this.adminService
+      .gather7Days()
+      .pipe(takeUntil(this.unsubscribeSubject))
+      .subscribe(() => {
+        setTimeout(() => {
+          window.location.reload();
+        }, 300);
+      });
+  }
 
   public onGatherMax() {
-    const confirmation = confirm(
-      'This action may take some time. Do you want to proceed?'
-    );
-
-    if (confirmation === true) {
     this.adminService
       .gatherMax()
       .pipe(takeUntil(this.unsubscribeSubject))
@@ -177,7 +183,6 @@ export class AdminOverviewComponent implements OnDestroy, OnInit {
         }, 300);
       });
   }
-  }
 
   public onGatherProfileData() {
     this.adminService
@@ -207,39 +212,15 @@ export class AdminOverviewComponent implements OnDestroy, OnInit {
     this.dataService
       .fetchAdminData()
       .pipe(takeUntil(this.unsubscribeSubject))
-      .subscribe(
+      .subscribe(({ exchangeRates, settings, transactionCount, userCount }) => {
-        ({
-          dataGatheringProgress,
-          exchangeRates,
-          lastDataGathering,
-          settings,
-          transactionCount,
-          userCount
-        }) => {
         this.coupons = (settings[PROPERTY_COUPONS] as Coupon[]) ?? [];
         this.customCurrencies = settings[PROPERTY_CURRENCIES] as string[];
-          this.dataGatheringProgress = dataGatheringProgress;
         this.exchangeRates = exchangeRates;
-
-          if (isValid(parseISO(lastDataGathering?.toString()))) {
-            this.lastDataGathering = formatDistanceToNowStrict(
-              new Date(lastDataGathering),
-              {
-                addSuffix: true
-              }
-            );
-          } else if (lastDataGathering === 'IN_PROGRESS') {
-            this.dataGatheringInProgress = true;
-          } else {
-            this.lastDataGathering = 'Starting soon...';
-          }
-
         this.transactionCount = transactionCount;
         this.userCount = userCount;
 
         this.changeDetectorRef.markForCheck();
-        }
+      });
-      );
   }
 
   private generateCouponCode(aLength: number) {

apps/client/src/app/components/admin-overview/admin-overview.html

@@ -19,37 +19,30 @@
           <div class="d-flex my-3">
             <div class="w-50" i18n>Data Gathering</div>
             <div class="w-50">
-              <div>
+              <div class="overflow-hidden">
-                <ng-container *ngIf="lastDataGathering"
-                  >{{ lastDataGathering }}</ng-container
-                >
-                <ng-container *ngIf="dataGatheringInProgress" i18n
-                  >In Progress ({{ dataGatheringProgress | percent : '1.2-2'
-                  }})</ng-container
-                >
-              </div>
-              <div class="mt-2 overflow-hidden">
                 <div class="mb-2">
                   <button
                     color="accent"
                     mat-flat-button
-                    (click)="onFlushCache()"
+                    (click)="onGather7Days()"
                   >
                     <ion-icon
                       class="mr-1"
-                      name="close-circle-outline"
+                      name="cloud-download-outline"
                     ></ion-icon>
-                    <span i18n>Reset Data Gathering</span>
+                    <span i18n>Gather Recent Data</span>
                   </button>
                 </div>
                 <div class="mb-2">
                   <button
-                    color="warn"
+                    color="accent"
                     mat-flat-button
-                    [disabled]="dataGatheringInProgress"
                     (click)="onGatherMax()"
                   >
-                    <ion-icon class="mr-1" name="warning-outline"></ion-icon>
+                    <ion-icon
+                      class="mr-1"
+                      name="cloud-download-outline"
+                    ></ion-icon>
                     <span i18n>Gather All Data</span>
                   </button>
                 </div>
@@ -58,7 +51,6 @@
                     class="mb-2 mr-2"
                     color="accent"
                     mat-flat-button
-                    [disabled]="dataGatheringInProgress"
                     (click)="onGatherProfileData()"
                   >
                     <ion-icon
@@ -97,7 +89,6 @@
                       *ngIf="customCurrencies.includes(exchangeRate.label2)"
                       class="mini-icon mx-1 no-min-width px-2"
                       mat-button
-                      [disabled]="dataGatheringInProgress"
                       (click)="onDeleteCurrency(exchangeRate.label2)"
                     >
                       <ion-icon name="trash-outline"></ion-icon>
@@ -109,7 +100,6 @@
                 <button
                   color="primary"
                   mat-flat-button
-                  [disabled]="dataGatheringInProgress"
                   (click)="onAddCurrency()"
                 >
                   <ion-icon class="mr-1" name="add-outline"></ion-icon>
@@ -126,7 +116,6 @@
                 <button
                   class="mini-icon mx-1 no-min-width px-2"
                   mat-button
-                  [disabled]="dataGatheringInProgress"
                   (click)="onDeleteSystemMessage()"
                 >
                   <ion-icon name="trash-outline"></ion-icon>
@@ -197,6 +186,15 @@
               </div>
             </div>
           </div>
+          <div class="d-flex my-3">
+            <div class="w-50" i18n>Housekeeping</div>
+            <div class="w-50">
+              <button color="warn" mat-flat-button (click)="onFlushCache()">
+                <ion-icon class="mr-1" name="close-circle-outline"></ion-icon>
+                <span i18n>Flush Cache</span>
+              </button>
+            </div>
+          </div>
         </mat-card-content>
       </mat-card>
     </div>

apps/client/src/app/components/home-market/home-market.component.ts

@@ -25,7 +25,7 @@ export class HomeMarketComponent implements OnDestroy, OnInit {
   public historicalData: HistoricalDataItem[];
   public info: InfoItem;
   public isLoading = true;
-  public readonly numberOfDays = 90;
+  public readonly numberOfDays = 180;
   public user: User;
 
   private unsubscribeSubject = new Subject<void>();

apps/client/src/app/components/portfolio-performance/portfolio-performance.component.ts

@@ -81,9 +81,11 @@ export class PortfolioPerformanceComponent implements OnChanges, OnInit {
   }
 
   public onShowErrors() {
-    const errorMessageParts = this.errors.map((error) => {
+    const errorMessageParts = ['Data Provider Errors for'];
-      return `${error.symbol} (${error.dataSource})`;
+
-    });
+    for (const error of this.errors) {
+      errorMessageParts.push(`${error.symbol} (${error.dataSource})`);
+    }
 
     alert(errorMessageParts.join('\n'));
   }

apps/client/src/app/pages/account/account-page.html

@@ -169,6 +169,10 @@
               ></mat-slide-toggle>
             </div>
           </div>
+          <div class="align-items-center d-flex mt-4 py-1">
+            <div class="pr-1 w-50" i18n>ID</div>
+            <div class="pl-1 w-50">{{ user?.id }}</div>
+          </div>
         </mat-card-content>
       </mat-card>
     </div>

apps/client/src/app/pages/accounts/accounts-page.component.ts

@@ -133,6 +133,11 @@ export class AccountsPageComponent implements OnDestroy, OnInit {
       .pipe(takeUntil(this.unsubscribeSubject))
       .subscribe({
         next: () => {
+          this.userService
+            .get(true)
+            .pipe(takeUntil(this.unsubscribeSubject))
+            .subscribe();
+
           this.fetchAccounts();
         }
       });
@@ -179,6 +184,11 @@ export class AccountsPageComponent implements OnDestroy, OnInit {
             .pipe(takeUntil(this.unsubscribeSubject))
             .subscribe({
               next: () => {
+                this.userService
+                  .get(true)
+                  .pipe(takeUntil(this.unsubscribeSubject))
+                  .subscribe();
+
                 this.fetchAccounts();
               }
             });
@@ -220,6 +230,11 @@ export class AccountsPageComponent implements OnDestroy, OnInit {
             .pipe(takeUntil(this.unsubscribeSubject))
             .subscribe({
               next: () => {
+                this.userService
+                  .get(true)
+                  .pipe(takeUntil(this.unsubscribeSubject))
+                  .subscribe();
+
                 this.fetchAccounts();
               }
             });

apps/client/src/app/pages/accounts/create-or-update-account-dialog/create-or-update-account-dialog.html

@@ -29,7 +29,7 @@
     </div>
     <div>
       <mat-form-field appearance="outline" class="w-100">
-        <mat-label i18n>Balance</mat-label>
+        <mat-label i18n>Cash Balance</mat-label>
         <input
           matInput
           name="balance"

apps/client/src/app/pages/admin/admin-page-routing.module.ts

@@ -1,5 +1,6 @@
 import { NgModule } from '@angular/core';
 import { RouterModule, Routes } from '@angular/router';
+import { AdminJobsComponent } from '@ghostfolio/client/components/admin-jobs/admin-jobs.component';
 import { AdminMarketDataComponent } from '@ghostfolio/client/components/admin-market-data/admin-market-data.component';
 import { AdminOverviewComponent } from '@ghostfolio/client/components/admin-overview/admin-overview.component';
 import { AdminUsersComponent } from '@ghostfolio/client/components/admin-users/admin-users.component';
@@ -14,6 +15,7 @@ const routes: Routes = [
     canActivate: [AuthGuard],
     children: [
       { path: '', redirectTo: 'overview', pathMatch: 'full' },
+      { path: 'jobs', component: AdminJobsComponent },
       { path: 'market-data', component: AdminMarketDataComponent },
       { path: 'overview', component: AdminOverviewComponent },
       { path: 'users', component: AdminUsersComponent }

apps/client/src/app/pages/admin/admin-page.html

@@ -5,7 +5,8 @@
     *ngFor="let link of [
   { iconName: 'reader-outline', path: 'overview' },
   { iconName: 'people-outline', path: 'users' },
-  { iconName: 'server-outline', path: 'market-data' }
+  { iconName: 'server-outline', path: 'market-data' },
+  { iconName: 'flash-outline', path: 'jobs' }
 ]"
     #rla="routerLinkActive"
     mat-tab-link

apps/client/src/app/pages/admin/admin-page.module.ts

@@ -4,6 +4,7 @@ import { MatButtonModule } from '@angular/material/button';
 import { MatCardModule } from '@angular/material/card';
 import { MatMenuModule } from '@angular/material/menu';
 import { MatTabsModule } from '@angular/material/tabs';
+import { GfAdminJobsModule } from '@ghostfolio/client/components/admin-jobs/admin-jobs.module';
 import { GfAdminMarketDataModule } from '@ghostfolio/client/components/admin-market-data/admin-market-data.module';
 import { GfAdminOverviewModule } from '@ghostfolio/client/components/admin-overview/admin-overview.module';
 import { GfAdminUsersModule } from '@ghostfolio/client/components/admin-users/admin-users.module';
@@ -19,6 +20,7 @@ import { AdminPageComponent } from './admin-page.component';
   imports: [
     AdminPageRoutingModule,
     CommonModule,
+    GfAdminJobsModule,
     GfAdminMarketDataModule,
     GfAdminOverviewModule,
     GfAdminUsersModule,

apps/client/src/app/pages/admin/admin-page.scss

@@ -11,6 +11,7 @@
   padding-bottom: constant(safe-area-inset-bottom);
 
   ::ng-deep {
+    gf-admin-jobs,
     gf-admin-market-data,
     gf-admin-overview,
     gf-admin-users {

apps/client/src/app/pages/blog/blog-page.html

@@ -14,7 +14,7 @@
                   <div class="h6 m-0 text-truncate">
                     First months in Open Source
                   </div>
-                  <div class="d-flex text-muted">05.01.2021</div>
+                  <div class="d-flex text-muted">05.01.2022</div>
                 </div>
                 <div class="align-items-center d-flex">
                   <ion-icon

apps/client/src/app/pages/features/features-page.html

@@ -4,14 +4,12 @@
       <h3 class="d-flex justify-content-center mb-3 text-center" i18n>
         Features
       </h3>
-      <mat-card class="mb-4">
+      <div class="mb-4">
-        <mat-card-content>
         <p>
           Check out the numerous features of <strong>Ghostfolio</strong> to
           manage your wealth.
         </p>
-        </mat-card-content>
+      </div>
-      </mat-card>
       <div class="row">
         <div class="col-xs-12 col-md-4 mb-3">
           <mat-card class="d-flex flex-column h-100">

apps/client/src/app/pages/portfolio/fire/fire-page.html

@@ -1,8 +1,22 @@
 <div class="container">
-  <div class="row">
+  <div class="row mb-5">
     <div class="col-lg">
       <h3 class="d-flex justify-content-center mb-3" i18n>FIRE</h3>
-      <div class="mb-5">
+      <div>
+        <h4 class="mb-3" i18n>Calculator</h4>
+        <gf-fire-calculator
+          [currency]="user?.settings?.baseCurrency"
+          [deviceType]="deviceType"
+          [fireWealth]="fireWealth?.toNumber()"
+          [hasPermissionToUpdateUserSettings]="hasPermissionToUpdateUserSettings"
+          [locale]="user?.settings?.locale"
+          [savingsRate]="user?.settings?.savingsRate"
+          (savingsRateChanged)="onSavingsRateChange($event)"
+        ></gf-fire-calculator>
+      </div>
+    </div>
+  </div>
+  <div>
     <h4 i18n>4% Rule</h4>
     <div *ngIf="isLoading">
       <ngx-skeleton-loader
@@ -51,18 +65,4 @@
       and a withdrawal rate of 4%.
     </div>
   </div>
-    </div>
-  </div>
-  <div>
-    <h4 class="mb-3" i18n>Calculator</h4>
-    <gf-fire-calculator
-      [currency]="user?.settings?.baseCurrency"
-      [deviceType]="deviceType"
-      [fireWealth]="fireWealth?.toNumber()"
-      [hasPermissionToUpdateUserSettings]="hasPermissionToUpdateUserSettings"
-      [locale]="user?.settings?.locale"
-      [savingsRate]="user?.settings?.savingsRate"
-      (savingsRateChanged)="onSavingsRateChange($event)"
-    ></gf-fire-calculator>
-  </div>
 </div>

apps/client/src/app/pages/pricing/pricing-page.scss

@@ -21,8 +21,4 @@
 
 :host-context(.is-dark-theme) {
   color: rgb(var(--light-primary-text));
-
-  a {
-    color: rgb(var(--light-primary-text));
-  }
 }

apps/client/src/app/services/admin.service.ts

@@ -1,13 +1,15 @@
-import { HttpClient } from '@angular/common/http';
+import { HttpClient, HttpParams } from '@angular/common/http';
 import { Injectable } from '@angular/core';
 import { UpdateMarketDataDto } from '@ghostfolio/api/app/admin/update-market-data.dto';
 import { IDataProviderHistoricalResponse } from '@ghostfolio/api/services/interfaces/interfaces';
 import { DATE_FORMAT } from '@ghostfolio/common/helper';
 import {
+  AdminJobs,
   AdminMarketDataDetails,
   UniqueAsset
 } from '@ghostfolio/common/interfaces';
 import { DataSource, MarketData } from '@prisma/client';
+import { JobStatus } from 'bull';
 import { format, parseISO } from 'date-fns';
 import { Observable, map } from 'rxjs';
 
@@ -17,6 +19,22 @@ import { Observable, map } from 'rxjs';
 export class AdminService {
   public constructor(private http: HttpClient) {}
 
+  public deleteJob(aId: string) {
+    return this.http.delete<void>(`/api/v1/admin/queue/job/${aId}`);
+  }
+
+  public deleteJobs({ status }: { status: JobStatus[] }) {
+    let params = new HttpParams();
+
+    if (status?.length > 0) {
+      params = params.append('status', status.join(','));
+    }
+
+    return this.http.delete<void>('/api/v1/admin/queue/job', {
+      params
+    });
+  }
+
   public deleteProfileData({ dataSource, symbol }: UniqueAsset) {
     return this.http.delete<void>(
       `/api/v1/admin/profile-data/${dataSource}/${symbol}`
@@ -42,12 +60,28 @@ export class AdminService {
       );
   }
 
+  public fetchJobs({ status }: { status?: JobStatus[] }) {
+    let params = new HttpParams();
+
+    if (status?.length > 0) {
+      params = params.append('status', status.join(','));
+    }
+
+    return this.http.get<AdminJobs>('/api/v1/admin/queue/job', {
+      params
+    });
+  }
+
+  public gather7Days() {
+    return this.http.post<void>('/api/v1/admin/gather', {});
+  }
+
   public gatherMax() {
-    return this.http.post<void>(`/api/v1/admin/gather/max`, {});
+    return this.http.post<void>('/api/v1/admin/gather/max', {});
   }
 
   public gatherProfileData() {
-    return this.http.post<void>(`/api/v1/admin/gather/profile-data`, {});
+    return this.http.post<void>('/api/v1/admin/gather/profile-data', {});
   }
 
   public gatherProfileDataBySymbol({ dataSource, symbol }: UniqueAsset) {

apps/client/src/app/services/user/user-store.actions.ts

@@ -1,4 +1,5 @@
 export enum UserStoreActions {
   GetUser = 'GET_USER',
+  Initialize = 'INITIALIZE',
   RemoveUser = 'REMOVE_USER'
 }

apps/client/src/app/services/user/user.service.ts

@@ -16,13 +16,13 @@ export class UserService extends ObservableStore<UserStoreState> {
   public constructor(private http: HttpClient) {
     super({ trackStateHistory: true });
 
-    this.setState({ user: undefined }, 'INIT_STATE');
+    this.setState({ user: undefined }, UserStoreActions.Initialize);
   }
 
-  public get() {
+  public get(force = false) {
     const state = this.getState();
 
-    if (state?.user) {
+    if (state?.user && force !== true) {
       // Get from cache
       return of(state.user);
     } else {

docker/docker-compose.build.yml

@@ -5,7 +5,7 @@ services:
     env_file:
       - ../.env
     environment:
-      DATABASE_URL: postgresql://user:password@postgres:5432/ghostfolio-db?sslmode=prefer
+      DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}?sslmode=prefer
       REDIS_HOST: 'redis'
       REDIS_PASSWORD: ${REDIS_PASSWORD}
     ports:

docker/docker-compose.yml

@@ -5,7 +5,7 @@ services:
     env_file:
       - ../.env
     environment:
-      DATABASE_URL: postgresql://user:password@postgres:5432/ghostfolio-db?sslmode=prefer
+      DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}?sslmode=prefer
       REDIS_HOST: 'redis'
       REDIS_PASSWORD: ${REDIS_PASSWORD}
     ports:

libs/common/src/lib/config.ts

@@ -1,4 +1,6 @@
 import { DataSource } from '@prisma/client';
+import { JobOptions, JobStatus } from 'bull';
+import ms from 'ms';
 
 import { ToggleOption } from './types';
 
@@ -43,19 +45,52 @@ export const warnColorRgb = {
 export const ASSET_SUB_CLASS_EMERGENCY_FUND = 'EMERGENCY_FUND';
 
 export const DATA_GATHERING_QUEUE = 'DATA_GATHERING_QUEUE';
+export const DATA_GATHERING_QUEUE_PRIORITY_LOW = Number.MAX_SAFE_INTEGER;
+export const DATA_GATHERING_QUEUE_PRIORITY_HIGH = 1;
 
 export const DEFAULT_DATE_FORMAT_MONTH_YEAR = 'MMM yyyy';
 
 export const GATHER_ASSET_PROFILE_PROCESS = 'GATHER_ASSET_PROFILE';
+export const GATHER_ASSET_PROFILE_PROCESS_OPTIONS: JobOptions = {
+  attempts: 10,
+  backoff: {
+    delay: ms('1 minute'),
+    type: 'exponential'
+  },
+  priority: DATA_GATHERING_QUEUE_PRIORITY_HIGH,
+  removeOnComplete: {
+    age: ms('2 weeks') / 1000
+  }
+};
+export const GATHER_HISTORICAL_MARKET_DATA_PROCESS =
+  'GATHER_HISTORICAL_MARKET_DATA';
+export const GATHER_HISTORICAL_MARKET_DATA_PROCESS_OPTIONS: JobOptions = {
+  attempts: 10,
+  backoff: {
+    delay: ms('1 minute'),
+    type: 'exponential'
+  },
+  priority: DATA_GATHERING_QUEUE_PRIORITY_LOW,
+  removeOnComplete: {
+    age: ms('2 weeks') / 1000
+  }
+};
 
 export const PROPERTY_BENCHMARKS = 'BENCHMARKS';
 export const PROPERTY_COUPONS = 'COUPONS';
 export const PROPERTY_CURRENCIES = 'CURRENCIES';
 export const PROPERTY_IS_READ_ONLY_MODE = 'IS_READ_ONLY_MODE';
-export const PROPERTY_LAST_DATA_GATHERING = 'LAST_DATA_GATHERING';
-export const PROPERTY_LOCKED_DATA_GATHERING = 'LOCKED_DATA_GATHERING';
 export const PROPERTY_SLACK_COMMUNITY_USERS = 'SLACK_COMMUNITY_USERS';
 export const PROPERTY_STRIPE_CONFIG = 'STRIPE_CONFIG';
 export const PROPERTY_SYSTEM_MESSAGE = 'SYSTEM_MESSAGE';
 
+export const QUEUE_JOB_STATUS_LIST = <JobStatus[]>[
+  'active',
+  'completed',
+  'delayed',
+  'failed',
+  'paused',
+  'waiting'
+];
+
 export const UNKNOWN_KEY = 'UNKNOWN';

libs/common/src/lib/helper.ts

@@ -77,6 +77,10 @@ export function getDateFormatString(aLocale?: string) {
     .join('');
 }
 
+export function getDateWithTimeFormatString(aLocale?: string) {
+  return `${getDateFormatString(aLocale)}, HH:mm:ss`;
+}
+
 export function getLocale() {
   return navigator.languages?.length
     ? navigator.languages[0]

libs/common/src/lib/interfaces/admin-data.interface.ts

@@ -1,7 +1,5 @@
 export interface AdminData {
-  dataGatheringProgress?: number;
   exchangeRates: { label1: string; label2: string; value: number }[];
-  lastDataGathering?: Date | 'IN_PROGRESS';
   settings: { [key: string]: boolean | object | string | string[] };
   transactionCount: number;
   userCount: number;

libs/common/src/lib/interfaces/admin-jobs.interface.ts

@@ -0,0 +1,16 @@
+import { Job, JobStatus } from 'bull';
+
+export interface AdminJobs {
+  jobs: (Pick<
+    Job<any>,
+    | 'attemptsMade'
+    | 'data'
+    | 'finishedOn'
+    | 'id'
+    | 'name'
+    | 'stacktrace'
+    | 'timestamp'
+  > & {
+    state: JobStatus | 'stuck';
+  })[];
+}

libs/common/src/lib/interfaces/index.ts

@@ -1,6 +1,7 @@
 import { Access } from './access.interface';
 import { Accounts } from './accounts.interface';
 import { AdminData } from './admin-data.interface';
+import { AdminJobs } from './admin-jobs.interface';
 import { AdminMarketDataDetails } from './admin-market-data-details.interface';
 import {
   AdminMarketData,
@@ -40,6 +41,7 @@ export {
   Access,
   Accounts,
   AdminData,
+  AdminJobs,
   AdminMarketData,
   AdminMarketDataDetails,
   AdminMarketDataItem,

libs/ui/src/lib/activities-filter/activities-filter.component.ts

@@ -126,7 +126,7 @@ export class ActivitiesFilterComponent implements OnChanges, OnDestroy {
 
           return filter;
         })
-        .sort((a, b) => a.label.localeCompare(b.label)),
+        .sort((a, b) => a.label?.localeCompare(b.label)),
       (filter) => {
         return filter.type;
       }
@@ -142,7 +142,7 @@ export class ActivitiesFilterComponent implements OnChanges, OnDestroy {
     }
 
     return filterGroups
-      .sort((a, b) => a.name.localeCompare(b.name))
+      .sort((a, b) => a.name?.localeCompare(b.name))
       .map((filterGroup) => {
         return {
           ...filterGroup,

libs/ui/src/lib/activities-table/activities-table.component.html

@@ -322,7 +322,7 @@
             (click)="onImport()"
           >
             <ion-icon class="mr-2" name="cloud-upload-outline"></ion-icon>
-            <span i18n>Import</span>
+            <span i18n>Import Activities</span>
           </button>
           <button
             *ngIf="hasPermissionToExportActivities"
@@ -332,7 +332,7 @@
             (click)="onExport()"
           >
             <ion-icon class="mr-2" name="cloud-download-outline"></ion-icon>
-            <span i18n>Export</span>
+            <span i18n>Export Activities</span>
           </button>
           <button
             *ngIf="hasPermissionToExportActivities"

libs/ui/src/lib/activities-table/activities-table.component.ts

@@ -192,11 +192,13 @@ export class ActivitiesTableComponent implements OnChanges, OnDestroy {
     activity: OrderWithAccount,
     fieldValueMap: { [id: string]: Filter } = {}
   ): Filter[] {
-    fieldValueMap[activity.Account?.id] = {
+    if (activity.Account?.id) {
-      id: activity.Account?.id,
+      fieldValueMap[activity.Account.id] = {
-      label: activity.Account?.name,
+        id: activity.Account.id,
+        label: activity.Account.name,
         type: 'ACCOUNT'
       };
+    }
 
     fieldValueMap[activity.SymbolProfile.currency] = {
       id: activity.SymbolProfile.currency,

libs/ui/src/lib/fire-calculator/fire-calculator.component.html

@@ -1,7 +1,7 @@
 <div class="container p-0">
   <div class="row">
     <div class="col-md-3">
-      <form class="" [formGroup]="calculatorForm">
+      <form class="mb-4" [formGroup]="calculatorForm">
         <!--<mat-form-field appearance="outline">
           <input formControlName="principalInvestmentAmount" matInput />
         </mat-form-field>-->

package.json

@@ -1,6 +1,6 @@
 {
   "name": "ghostfolio",
-  "version": "1.155.0",
+  "version": "1.158.1",
   "homepage": "https://ghostfol.io",
   "license": "AGPL-3.0",
   "scripts": {
@@ -73,8 +73,8 @@
     "@nestjs/serve-static": "2.2.2",
     "@nrwl/angular": "14.1.4",
     "@prisma/client": "3.14.0",
-    "@simplewebauthn/browser": "4.1.0",
+    "@simplewebauthn/browser": "5.2.1",
-    "@simplewebauthn/server": "4.1.0",
+    "@simplewebauthn/server": "5.2.1",
     "@stripe/stripe-js": "1.22.0",
     "alphavantage": "2.2.0",
     "angular-material-css-vars": "3.0.0",
@@ -84,7 +84,7 @@
     "bull": "4.8.2",
     "cache-manager": "3.4.3",
     "cache-manager-redis-store": "2.0.0",
-    "chart.js": "3.7.0",
+    "chart.js": "3.8.0",
     "chartjs-adapter-date-fns": "2.0.0",
     "chartjs-plugin-datalabels": "2.0.0",
     "cheerio": "1.0.0-rc.6",
@@ -93,9 +93,8 @@
     "color": "4.0.1",
     "countries-list": "2.6.1",
     "countup.js": "2.0.7",
-    "cryptocurrencies": "7.0.0",
     "date-fns": "2.22.1",
-    "envalid": "7.2.1",
+    "envalid": "7.3.1",
     "google-spreadsheet": "3.2.0",
     "http-status-codes": "2.2.0",
     "ionicons": "5.5.1",
@@ -139,7 +138,7 @@
     "@nrwl/nx-cloud": "14.0.3",
     "@nrwl/storybook": "14.1.4",
     "@nrwl/workspace": "14.1.4",
-    "@simplewebauthn/typescript-types": "4.0.0",
+    "@simplewebauthn/typescript-types": "5.2.1",
     "@storybook/addon-essentials": "6.4.22",
     "@storybook/angular": "6.4.22",
     "@storybook/builder-webpack5": "6.4.22",
@@ -159,7 +158,6 @@
     "@typescript-eslint/parser": "5.4.0",
     "codelyzer": "6.0.1",
     "cypress": "6.2.1",
-    "dotenv": "10.0.0",
     "eslint": "8.3.0",
     "eslint-config-prettier": "8.3.0",
     "eslint-plugin-cypress": "2.12.1",

yarn.lock

@@ -3554,35 +3554,34 @@
     node-addon-api "^3.2.1"
     node-gyp-build "^4.3.0"
 
-"@peculiar/asn1-android@^2.0.38":
+"@peculiar/asn1-android@^2.1.7":
-  version "2.0.38"
+  version "2.1.8"
-  resolved "https://registry.yarnpkg.com/@peculiar/asn1-android/-/asn1-android-2.0.38.tgz#193281f5a232e323d6f2c069c7a8e8e8f4a994bd"
+  resolved "https://registry.yarnpkg.com/@peculiar/asn1-android/-/asn1-android-2.1.8.tgz#64b6da2b5a03ddb86bcc9061d981be7ba811069d"
-  integrity sha512-krWyggV6FgYf3fEPKVNjHVecLcQWlAu3/YhOyN+/L43dNKcsmqiEvuhqplh3aiXF62Ds0pqzqttWmdvoVqmSVQ==
+  integrity sha512-SgtOvNES2Aex5rafRlQiaAbWd38hMLwwtQL13ndVhDN1/NYxPF3VgeJWv3KKRY4uFh9VXvF6NuRfEcrSX5UWiQ==
   dependencies:
-    "@peculiar/asn1-schema" "^2.0.38"
+    "@peculiar/asn1-schema" "^2.1.8"
-    asn1js "^2.1.1"
+    asn1js "^3.0.4"
-    tslib "^2.3.0"
+    tslib "^2.4.0"
 
-"@peculiar/asn1-schema@^2.0.38":
+"@peculiar/asn1-schema@^2.1.7", "@peculiar/asn1-schema@^2.1.8":
-  version "2.0.38"
+  version "2.1.8"
-  resolved "https://registry.yarnpkg.com/@peculiar/asn1-schema/-/asn1-schema-2.0.38.tgz#98b6f12daad275ecd6774dfe31fb62f362900412"
+  resolved "https://registry.yarnpkg.com/@peculiar/asn1-schema/-/asn1-schema-2.1.8.tgz#552300a1ed7991b22c9abf789a3920a3cb94c26b"
-  integrity sha512-zZ64UpCTm9me15nuCpPgJghSdbEm8atcDQPCyK+bKXjZAQ1735NCZXCSCfbckbQ4MH36Rm9403n/qMq77LFDzQ==
+  integrity sha512-u34H/bpqCdDuqrCVZvH0vpwFBT/dNEdNY+eE8u4IuC26yYnhDkXF4+Hliqca88Avbb7hyN2EF/eokyDdyS7G/A==
   dependencies:
-    "@types/asn1js" "^2.0.2"
+    asn1js "^3.0.4"
-    asn1js "^2.1.1"
+    pvtsutils "^1.3.2"
-    pvtsutils "^1.2.0"
+    tslib "^2.4.0"
-    tslib "^2.3.0"
 
-"@peculiar/asn1-x509@^2.0.38":
+"@peculiar/asn1-x509@^2.1.7":
-  version "2.0.38"
+  version "2.1.8"
-  resolved "https://registry.yarnpkg.com/@peculiar/asn1-x509/-/asn1-x509-2.0.38.tgz#7ff3b5478d9c3784f0eb2fbe7693509da9de0a43"
+  resolved "https://registry.yarnpkg.com/@peculiar/asn1-x509/-/asn1-x509-2.1.8.tgz#b67317ba1ee33c758ad7c6145dbaa1ddef4f1913"
-  integrity sha512-10aK9fSxlc1DK9nEcwh+WPFNhAheXSE9RbI5MyS7FdBhgq+Mz4Z9JqFfaBZm1Qp+5mPtUMOP6cXVo7aaYlgq7A==
+  integrity sha512-asAcoeZ+bjy/4/lf6gbMlfmywHpxLBa7LBE4pPCzSAKBM0IHXWa7bqsDyshtywzLW+VpA+G2m0Fs7Lt7Woh7RA==
   dependencies:
-    "@peculiar/asn1-schema" "^2.0.38"
+    "@peculiar/asn1-schema" "^2.1.8"
-    asn1js "^2.1.1"
+    asn1js "^3.0.4"
     ipaddr.js "^2.0.1"
-    pvtsutils "^1.2.0"
+    pvtsutils "^1.3.2"
-    tslib "^2.3.0"
+    tslib "^2.4.0"
 
 "@phenomnomnominal/tsquery@4.1.1":
   version "4.1.1"
@@ -3629,32 +3628,33 @@
     "@angular-devkit/schematics" "13.3.5"
     jsonc-parser "3.0.0"
 
-"@simplewebauthn/browser@4.1.0":
+"@simplewebauthn/browser@5.2.1":
-  version "4.1.0"
+  version "5.2.1"
-  resolved "https://registry.yarnpkg.com/@simplewebauthn/browser/-/browser-4.1.0.tgz#3e7fd66729405d6a2a2a187c93577b90a8e41786"
+  resolved "https://registry.yarnpkg.com/@simplewebauthn/browser/-/browser-5.2.1.tgz#569252a9f235a99aae90c4d1cc6c441f42637b8e"
-  integrity sha512-tIsEfShC1rrqrsNb44tOFuSriAFCz4tkdDnCjHfn2rYxgz+t+yqEvuIRfJHQpFrWSnZPdsjrAHtasj6lzfGI6w==
+  integrity sha512-TxL3OPHJf57hmnfQoF3zRIQWEdsJLxrA9NcGdRK0sB/h3jd13kpGQonBtMnj4YBQnWTtRDZ804wlpI9IEMaJ9g==
 
-"@simplewebauthn/server@4.1.0":
+"@simplewebauthn/server@5.2.1":
-  version "4.1.0"
+  version "5.2.1"
-  resolved "https://registry.yarnpkg.com/@simplewebauthn/server/-/server-4.1.0.tgz#9ad2e32cffa83833ff8a633775b2ace5e6926fa0"
+  resolved "https://registry.yarnpkg.com/@simplewebauthn/server/-/server-5.2.1.tgz#49038d2951ad2ac065bdf8342fdb13f78ee4df1c"
-  integrity sha512-52X5/U+5Fo0XYG1TuBBGgG0ap9c0ffpeq0GZfFio/DZDW4He0Arb7Q/XkHw96JK0X1sfRKNmnfC+NImplvIimA==
+  integrity sha512-+CQ8oJf9Io8y4ReYLagX5JG9ShntIkdeCPkMoyHLBSRPlNY0N/Yv3Iun4YPQ8d4LJUU9f8S1eD5bibIEMjWDRg==
   dependencies:
-    "@peculiar/asn1-android" "^2.0.38"
+    "@peculiar/asn1-android" "^2.1.7"
-    "@peculiar/asn1-schema" "^2.0.38"
+    "@peculiar/asn1-schema" "^2.1.7"
-    "@peculiar/asn1-x509" "^2.0.38"
+    "@peculiar/asn1-x509" "^2.1.7"
-    "@simplewebauthn/typescript-types" "^4.0.0"
+    "@simplewebauthn/typescript-types" "^5.2.1"
     base64url "^3.0.1"
     cbor "^5.1.0"
+    debug "^4.3.2"
     elliptic "^6.5.3"
     jsrsasign "^10.4.0"
     jwk-to-pem "^2.0.4"
     node-fetch "^2.6.0"
     node-rsa "^1.1.1"
 
-"@simplewebauthn/typescript-types@4.0.0", "@simplewebauthn/typescript-types@^4.0.0":
+"@simplewebauthn/typescript-types@5.2.1", "@simplewebauthn/typescript-types@^5.2.1":
-  version "4.0.0"
+  version "5.2.1"
-  resolved "https://registry.yarnpkg.com/@simplewebauthn/typescript-types/-/typescript-types-4.0.0.tgz#46ae4e69cb07305c57093a3ed99555437dfe0d49"
+  resolved "https://registry.yarnpkg.com/@simplewebauthn/typescript-types/-/typescript-types-5.2.1.tgz#a8229ce4f71be7edafe3bfdce062b332ef494f0d"
-  integrity sha512-jqQ0bCeBO96CytB397vSrQ8ipozQzAmI57izA7izyglyu35JBV90I7+75fSX+ZGNHmMwDNnA3EGYtBLOIpkJEg==
+  integrity sha512-t/NzbjaD0zu4ivUmiof2cPA8X5LHhFX+DflBBl71/dzEhl15qepDI2rxWdjB+Hc0FfOT1fBQnb1uP19fPcDUiA==
 
 "@sinonjs/commons@^1.7.0":
   version "1.8.3"
@@ -4804,11 +4804,6 @@
   resolved "https://registry.yarnpkg.com/@tootallnate/once/-/once-2.0.0.tgz#f544a148d3ab35801c1f633a7441fd87c2e484bf"
   integrity sha512-XCuKFP5PS55gnMVu3dty8KPatLqUoy/ZYzDzAGCQ8JNFCkLXzmI7vNHCR+XpbZaMWQK/vQubr7PkYq8g470J/A==
 
-"@types/asn1js@^2.0.2":
-  version "2.0.2"
-  resolved "https://registry.yarnpkg.com/@types/asn1js/-/asn1js-2.0.2.tgz#bb1992291381b5f06e22a829f2ae009267cdf8c5"
-  integrity sha512-t4YHCgtD+ERvH0FyxvNlYwJ2ezhqw7t+Ygh4urQ7dJER8i185JPv6oIM3ey5YQmGN6Zp9EMbpohkjZi9t3UxwA==
-
 "@types/babel__core@^7.0.0", "@types/babel__core@^7.1.14":
   version "7.1.15"
   resolved "https://registry.yarnpkg.com/@types/babel__core/-/babel__core-7.1.15.tgz#2ccfb1ad55a02c83f8e0ad327cbc332f55eb1024"
@@ -6367,12 +6362,14 @@ asn1@^0.2.4, asn1@~0.2.3:
   dependencies:
     safer-buffer "~2.1.0"
 
-asn1js@^2.1.1:
+asn1js@^3.0.4:
-  version "2.1.1"
+  version "3.0.5"
-  resolved "https://registry.yarnpkg.com/asn1js/-/asn1js-2.1.1.tgz#bb3896191ebb5fb1caeda73436a6c6e20a2eedff"
+  resolved "https://registry.yarnpkg.com/asn1js/-/asn1js-3.0.5.tgz#5ea36820443dbefb51cc7f88a2ebb5b462114f38"
-  integrity sha512-t9u0dU0rJN4ML+uxgN6VM2Z4H5jWIYm0w8LsZLzMJaQsgL3IJNbxHgmbWDvJAwspyHpDFuzUaUFh4c05UB4+6g==
+  integrity sha512-FVnvrKJwpt9LP2lAMl8qZswRNm3T4q9CON+bxldk2iwk3FFpuwhx2FfinyitizWHsVYyaY+y5JzDR0rCMV5yTQ==
   dependencies:
-    pvutils latest
+    pvtsutils "^1.3.2"
+    pvutils "^1.1.3"
+    tslib "^2.4.0"
 
 assert-plus@1.0.0, assert-plus@^1.0.0:
   version "1.0.0"
@@ -7387,10 +7384,10 @@ chardet@^0.7.0:
   resolved "https://registry.yarnpkg.com/chardet/-/chardet-0.7.0.tgz#90094849f0937f2eedc2425d0d28a9e5f0cbad9e"
   integrity sha512-mT8iDcrh03qDGRRmoA2hmBJnxpllMR+0/0qlzjqZES6NdiWDcZkCNAk4rPFZ9Q85r27unkiNNg8ZOiwZXBHwcA==
 
-chart.js@3.7.0:
+chart.js@3.8.0:
-  version "3.7.0"
+  version "3.8.0"
-  resolved "https://registry.yarnpkg.com/chart.js/-/chart.js-3.7.0.tgz#7a19c93035341df801d613993c2170a1fcf1d882"
+  resolved "https://registry.yarnpkg.com/chart.js/-/chart.js-3.8.0.tgz#c6c14c457b9dc3ce7f1514a59e9b262afd6f1a94"
-  integrity sha512-31gVuqqKp3lDIFmzpKIrBeum4OpZsQjSIAqlOpgjosHDJZlULtvwLEZKtEhIAZc7JMPaHlYMys40Qy9Mf+1AAg==
+  integrity sha512-cr8xhrXjLIXVLOBZPkBZVF6NDeiVIrPLHcMhnON7UufudL+CNeRrD+wpYanswlm8NpudMdrt3CHoLMQMxJhHRg==
 
 chartjs-adapter-date-fns@2.0.0:
   version "2.0.0"
@@ -8204,11 +8201,6 @@ crypto-browserify@^3.11.0:
     randombytes "^2.0.0"
     randomfill "^1.0.3"
 
-cryptocurrencies@7.0.0:
-  version "7.0.0"
-  resolved "https://registry.yarnpkg.com/cryptocurrencies/-/cryptocurrencies-7.0.0.tgz#4f8fc826a63c14a1294e0b401bb69503506f98a5"
-  integrity sha512-jCA+ykHJg0BAH4eR3T5uhZJ31kp9JrZZqn7Cga4tUgONSPdg4MGLv0SuL272llVXfUgvZZX2lYM4ZjJa/+hjYw==
-
 css-blank-pseudo@^3.0.2:
   version "3.0.3"
   resolved "https://registry.yarnpkg.com/css-blank-pseudo/-/css-blank-pseudo-3.0.3.tgz#36523b01c12a25d812df343a32c322d2a2324561"
@@ -9015,10 +9007,10 @@ env-paths@^2.2.0:
   resolved "https://registry.yarnpkg.com/env-paths/-/env-paths-2.2.1.tgz#420399d416ce1fbe9bc0a07c62fa68d67fd0f8f2"
   integrity sha512-+h1lkLKhZMTYjog1VEpJNG7NZJWcuc2DDk/qsqSTRRCOXiLjeQ1d1/udrUGhqMxUgAlwKNZ0cf2uqan5GLuS2A==
 
-envalid@7.2.1:
+envalid@7.3.1:
-  version "7.2.1"
+  version "7.3.1"
-  resolved "https://registry.yarnpkg.com/envalid/-/envalid-7.2.1.tgz#7e9e62f3bc1ed209517f65b563e24c7b79c9793b"
+  resolved "https://registry.yarnpkg.com/envalid/-/envalid-7.3.1.tgz#5bf6bbb4effab2d64a1991d8078b4ae38924f0d2"
-  integrity sha512-NU0ty82LSvHF+Uio9cLNKhrDyivFv7GSvhOu91WbtOOyNKRzXWeDZaopldXJkGBAZ5UuquqXp6VBUXuTfXrUrw==
+  integrity sha512-KL1YRwn8WcoF/Ty7t+yLLtZol01xr9ZJMTjzoGRM8NaSU+nQQjSWOQKKJhJP2P57bpdakJ9jbxqQX4fGTOicZg==
   dependencies:
     tslib "2.3.1"
 
@@ -15820,17 +15812,17 @@ punycode@^2.1.0, punycode@^2.1.1:
   resolved "https://registry.yarnpkg.com/punycode/-/punycode-2.1.1.tgz#b58b010ac40c22c5657616c8d2c2c02c7bf479ec"
   integrity sha512-XRsRjdf+j5ml+y/6GKHPZbrF/8p2Yga0JPtdqTIY2Xe5ohJPD9saDJJLPvp9+NSBprVvevdXZybnj2cv8OEd0A==
 
-pvtsutils@^1.2.0:
+pvtsutils@^1.3.2:
-  version "1.2.0"
+  version "1.3.2"
-  resolved "https://registry.yarnpkg.com/pvtsutils/-/pvtsutils-1.2.0.tgz#619e4767093d23cd600482600c16f4c36d3025bb"
+  resolved "https://registry.yarnpkg.com/pvtsutils/-/pvtsutils-1.3.2.tgz#9f8570d132cdd3c27ab7d51a2799239bf8d8d5de"
-  integrity sha512-IDefMJEQl7HX0FP2hIKJFnAR11klP1js2ixCrOaMhe3kXFK6RQ2ABUCuwWaaD4ib0hSbh2fGTICvWJJhDfNecA==
+  integrity sha512-+Ipe2iNUyrZz+8K/2IOo+kKikdtfhRKzNpQbruF2URmqPtoqAs8g3xS7TJvFF2GcPXjh7DkqMnpVveRFq4PgEQ==
   dependencies:
-    tslib "^2.2.0"
+    tslib "^2.4.0"
 
-pvutils@latest:
+pvutils@^1.1.3:
-  version "1.0.17"
+  version "1.1.3"
-  resolved "https://registry.yarnpkg.com/pvutils/-/pvutils-1.0.17.tgz#ade3c74dfe7178944fe44806626bd2e249d996bf"
+  resolved "https://registry.yarnpkg.com/pvutils/-/pvutils-1.1.3.tgz#f35fc1d27e7cd3dfbd39c0826d173e806a03f5a3"
-  integrity sha512-wLHYUQxWaXVQvKnwIDWFVKDJku9XDCvyhhxoq8dc5MFdIlRenyPI9eSfEtcvgHgD7FlvCyGAlWgOzRnZD99GZQ==
+  integrity sha512-pMpnA0qRdFp32b1sJl1wOJNxZLQ2cbQx+k6tjNtZ8CpvVhNqEPRgivZ2WOUev2YMajecdH7ctUPDvEe87nariQ==
 
 qs@6.7.0:
   version "6.7.0"
@@ -18106,7 +18098,7 @@ tslib@2.0.0:
   resolved "https://registry.yarnpkg.com/tslib/-/tslib-2.0.0.tgz#18d13fc2dce04051e20f074cc8387fd8089ce4f3"
   integrity sha512-lTqkx847PI7xEDYJntxZH89L2/aXInsyF2luSafe/+0fHOMjlBNXdH6th7f70qxLDhul7KZK0zC8V5ZIyHl0/g==
 
-tslib@2.3.1, tslib@^2.0.0, tslib@^2.0.3, tslib@^2.1.0, tslib@^2.2.0, tslib@^2.3.0, tslib@^2.3.1:
+tslib@2.3.1, tslib@^2.0.0, tslib@^2.0.3, tslib@^2.1.0, tslib@^2.3.0, tslib@^2.3.1:
   version "2.3.1"
   resolved "https://registry.yarnpkg.com/tslib/-/tslib-2.3.1.tgz#e8a335add5ceae51aa261d32a490158ef042ef01"
   integrity sha512-77EbyPPpMz+FRFRuAFlWMtmgUWGe9UOG2Z25NqCwiIjRhOf5iKGuzSe5P2w1laq+FkRy4p+PCuVkJSGkzTEKVw==
@@ -18116,6 +18108,11 @@ tslib@^1.10.0, tslib@^1.8.1, tslib@^1.9.0:
   resolved "https://registry.yarnpkg.com/tslib/-/tslib-1.14.1.tgz#cf2d38bdc34a134bcaf1091c41f6619e2f672d00"
   integrity sha512-Xni35NKzjgMrwevysHTCArtLDpPvye8zV/0E4EyYn43P7/7qvQwPh9BGkHewbMulVntbigmcT7rdX3BNo9wRJg==
 
+tslib@^2.4.0:
+  version "2.4.0"
+  resolved "https://registry.yarnpkg.com/tslib/-/tslib-2.4.0.tgz#7cecaa7f073ce680a05847aa77be941098f36dc3"
+  integrity sha512-d6xOpEDfsi2CZVlPQzGeux8XMwLT9hssAsaPYExaQMuYskwb+x1x7J371tWlbBdWHroy99KnVB6qIkUbs5X3UQ==
+
 tslib@~2.1.0:
   version "2.1.0"
   resolved "https://registry.yarnpkg.com/tslib/-/tslib-2.1.0.tgz#da60860f1c2ecaa5703ab7d39bc05b6bf988b97a"